# Robots.txt of dmoz.de # Last update: 20150719 # # . at end of some partial paths is intentional. # # NOTE: # Please do not crawl us faster than 1 hit/second # User-agent: * Disallow: /f Disallow: /ini Disallow: /index.h Disallow: /robots.txt. Disallow: /index.2