Robots

Cookbook /
: BlockCrawler Redirect web crawlers to different pages (missing script)
: ControllingWebRobots How to control web robots or bots trying to scan files (stable)
: GoogleSitemaps How to submit a complete list of web pages to google
PmWiki /
: Robots Setting available to control robots
PmWikiDe /
: Robots Umgang mit Web-Robotern, Web-Spidern und Suchmaschinen