User-agent: * Disallow: /administrator/ Disallow: /cache/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /libraries/ Disallow: /media/ Disallow: /modules/ Disallow: /plugins/ Disallow: /templates/ Disallow: /tmp/ Disallow: /xmlrpc/ User-agent: Googlebot Disallow: User-agent: Googlebot-Image Disallow: User-agent: Googlebot-Mobile Disallow: User-agent: Mediapartners-Google Disallow: User-agent: Slurp Disallow: User-agent: Yahoo-slurp Disallow: User-agent: searchpreview Disallow: User-agent: MSNBOT Allow: / User-agent: Teoma Disallow: user-agent: Feedfetcher-Google Disallow: User-agent: ia_archiver Disallow: User-agent: psbot/0.1 Disallow: / User-agent: Twiceler www.cuill.com/robots.html Disallow: / User-agent: Twiceler-0.9 http://www.cuill.com/twiceler/robot.html Disallow: / User-agent: Dwaarbot+(dwaarbot@dwaar.com) Disallow: / User-agent: CazoodleBot Disallow: / User-agent: GurujiBot Disallow: / User-agent: Giga­mega.bot/1.0; +http://www.giga­mega.net/bot.html Disallow: / User-agent: MJ12bot Disallow: / User-agent: Baidu Disallow: / User-agent: Speedy Disallow: / User-agent: IIITBOT/1.1 (Indian Language Web Search Engine; http://webkhoj.iiit.net; pvvpr at iiit dot ac dot in) Disallow: / User-agent: boitho.com-dc Disallow: / User-agent: ccbot Disallow: / User-agent: seekbot Disallow: / User-agent: Pete-Spider Light Disallow: / User-agent: AskJeeves Disallow: User-agent: Yandex Disallow: / User-agent: baiduspider Disallow: / User-agent: naverbot Disallow: / User-agent: yeti Disallow: / User-agent: twiceler Disallow: / User-agent: Gigabot Disallow: / User-agent: Scrubby Disallow: / User-agent: Robozilla Disallow: User-agent: Nutch Disallow: / User-agent: * sitemap: http://www.infoog.com/index.php?option=com_xmap&sitemap=1&view=xml&no_html=1 Crawl-delay: 30