Robots Text File

How to make search engines ignore parts of your site.

Place a plain text file in the root of your site called robots.txt (eg, http://example.com/robots.txt) with a list of the directories to ignore as below.

# sample robots.txt
User-agent: *
Disallow: /blah/blah/ # use hash to rem
Disallow: /tmp/
Disallow: /cgi-bin/

See robotstxt.org for more information.

Home > Robots Text File