# $Id: robots.txt,v 1.8 2007/03/25 20:05:19 dries Exp $ # # robots.txt # # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # # This file will be ignored unless it is at the root of your host: # Used: http://example.com/robots.txt # Ignored: http://example.com/site/robots.txt # # For more information about the robots.txt standard, see: # http://www.robotstxt.org/wc/robots.html # # For syntax checking, see: # http://www.sxw.org.uk/computing/robots/check.html # # Only Googlebot, Yahoo! (Slurp) and Ask (Teoma) support Allow # Only Googlebot, MSNbot and Yahoo! (Slurp) support wildcards # Only Ask (Teoma), MSNbot and Yahoo! (Slurp) support crawl delays # # Google, Microsoft, Yahoo and Ask support sitemap auto-discovery User-agent: * Disallow: /archives Disallow: /cgi-bin Disallow: /feeder Disallow: /guestbook Disallow: /mint Disallow: /scgi-bin Disallow: /test Disallow: /wp Disallow: /newblog Disallow: /category Disallow: /page Disallow: /tag # Disallow: /*/trackback # Disallow: /*/feed Disallow: /*?* Disallow: /*? Allow: /wp/wp-content/uploads # Google Image User-agent: Googlebot-Image Disallow: Allow: /* # Google AdSense User-agent: Mediapartners-Google* Disallow: Allow: /* # Internet Archiver Wayback Machine User-agent: ia_archiver Disallow: / # digg mirror User-agent: duggmirror Disallow: / # Majestic SEO User-Agent: MJ12bot Disallow: # BEGIN XML-SITEMAP-PLUGIN Sitemap: http://www.oriste.com/sitemap.xml.gz # END XML-SITEMAP-PLUGIN