# ////////////////////////////////////////////////////////////////////////////// # File name: http://www.s-kanava.fi/robots.txt # Description Robots eclusion standard file for search engines. For more # information about robots.txt please refer to # http://en.wikipedia.org/wiki/Robots.txt. # # Version no: 1.0.2 # Date Created: 2008/01/02 # Last Modified Date: 2010/06/07 # Modification Log: # 1.0.0 \ 2008/01/02 \ MRa \ initial version # 1.0.1 \ 2010/03/26 \ MRa \ Deleted unnecessary Disallow entries (ID 226) # 1.0.2 \ 2010/06/07 \ MRa \ Corrected the sitemap location url (ID 203) # ////////////////////////////////////////////////////////////////////////////// # Apply these settings to all robots User-agent: * # List here the directories or files where the crawlers are adviced not to # enter Disallow: /resources/ Disallow: /getbinary/ # set to the number of seconds to wait between successive requests to the same # server Crawl-delay: 1 # maximum rate is one page every 2 seconds Request-rate: 1/2 # only visit between 18:00 (06:00 PM) and 08:00 AM UT (GMT) Visit-time: 1800-0800 # Sitemap location. The Sitemap parameter is supported by major crawlers # (including Google, Yahoo, MSN, Ask). Sitemaps specifies the location of the # site's list of URLs. This parameter is independent from User-agent parameter # so it can be placed anywhere in the file. Sitemap: http://www.s-kanava.fi/sitemap.xml.gz