# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-agent: * Disallow: /*/poll_avatar Disallow: /*/poll_banner Disallow: /*?*canvas_subclass= Disallow: /*?*color= Disallow: /*?*gender= Disallow: /*?*product_id= Disallow: /*?*query= Disallow: /*?*sort= Disallow: /*?*scroll_id= Disallow: /*==$ Disallow: /404 Disallow: /422 Disallow: /500 Disallow: /503 Disallow: /cart Disallow: /new_campaign Disallow: /my-store Disallow: /checkout Disallow: /designs/*/edit Disallow: /following Disallow: /header Disallow: /users/password Disallow: /cdn-cgi/ User-agent: Pinterest/0.2 (+https://www.pinterest.com/bot.html) Crawl-delay: 1 User-agent: Mozilla/5.0 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html) Crawl-delay: 1 User-agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html) Crawl-delay: 1