# Text file for web spiders - Created Nk 26-Mar-2005 # Allow all user agents, but exclude them from /cgi-bin User-agent: * Disallow: /cgi-bin