# # This is a robots.txt file that can be used to indicate to # WWW robots which parts of a web site should not be accessed. # # 1. Uncomment following two lines by removing "#" at the beginning to # indicate that all robots should stay away from the /cgi-bin/ folder User-agent: * Disallow: /cgi-bin/ Disallow: /Connections/ Disallow: /live/ Disallow: /new/ Disallow: /nuusbrief/ Disallow: /preke_audio/ Disallow: /test/ # 2. Following two lines indicates that robots named "WWW_Robot_Name" # should stay away from this site. #User-agent: WWW_Robot_Name #Disallow: / # 3. Following two lines indicates that robots named "WWW_Wanderer_Name" # should not index files in the folder /private/ #User-agent: WWW_Wanderer_Name #Disallow: /private/