# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-Agent: * Disallow: /places/search Disallow: /city/ Disallow: *?*utm_source= Disallow: *?*utm_medium= Disallow: *?*utm_term= Disallow: *?*utm_content= Disallow: *?*utm_campaign= Disallow: *?*order= Disallow: */new$ Disallow: */new/$ Disallow: /places/*/profile/* Allow: /