# robots.txt automatically generated by PrestaShop e-commerce open-source solution # http://www.prestashop.com - http://www.prestashop.com/forums # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # For more information about the robots.txt standard, see: # http://www.robotstxt.org/robotstxt.html User-agent: * Crawl-delay: 60 Disallow: / ## DENIED BOTS ## - Bots which we do not allow at all, because they are bad, or useless. ############################################################################### # Name : panscient.com # Info : panscient.com # Header: panscient.com User-agent: panscient.com Disallow: / # Name : sqlmap # Info : # Header: sqlmap/1.3.7.38#dev (http://sqlmap.org) User-agent: sqlmap Disallow: / # Name : BUbiNG # Info : # Header: BUbiNG (+http://law.di.unimi.it/BUbiNG.html) User-agent: BUbiNG Disallow: / # Name : SiteExplorer # Info : http://siteexplorer.info/Backlink-Checker-Spider/ # Header: Mozilla/5.0 (compatible; SiteExplorer/1.1b; +http://siteexplorer.info/Backlink-Checker-Spider/) User-agent: SiteExplorer Disallow: / # Name : 360Spider # Info : # Header: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider User-agent: 360Spider Disallow: / # Name : cognitiveseo # Info : http://cognitiveseo.com/bot.html # Header: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 - James BOT - WebCrawler http://cognitiveseo.com/bot.html User-agent: cognitiveseo Disallow: / # Name : MJ12bot # Info : http://mj12bot.com/ # Header: Mozilla/5.0 (compatible; MJ12bot/v1.4.6; http://mj12bot.com/) User-agent: MJ12bot Disallow: / # Name : rogerbot # Info : http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, # Header: rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler+pr2-crawler-06@moz.com) User-agent: rogerbot Disallow: / # Name : Scrapy # Info : http://scrapy.org # Header: Scrapy/1.2.1 (+http://scrapy.org) User-agent: Scrapy Disallow: / # Name : Screaming # Info : # Header: Screaming Frog SEO Spider/5.1 User-agent: Screaming Disallow: / # Name : SemrushBot # Info : http://www.semrush.com/bot.html # Header: Mozilla/5.0 (compatible; SemrushBot/1.1~bl; +http://www.semrush.com/bot.html) User-agent: SemrushBot Disallow: / # Name : SMTBot # Info : http://www.similartech.com/smtbot # Header: Mozilla/5.0 (compatible; SMTBot/1.0; http://www.similartech.com/smtbot) User-agent: SMTBot Disallow: / # Name : spbot # Info : http://OpenLinkProfiler.org/bot # Header: Mozilla/5.0 (compatible; spbot/4.4.0; +http://OpenLinkProfiler.org/bot ) User-agent: spbot Disallow: / # Name : vebidoobot # Info : # Header: vebidoobot User-agent: vebidoobot Disallow: / # Name : YisouSpider # Info : # Header: YisouSpider User-agent: YisouSpider Disallow: / # Name : ZoomBot # Info : http://suite.seozoom.it/bot.html # Header: ZoomBot (Linkbot 1.0 http://suite.seozoom.it/bot.html) User-agent: ZoomBot Disallow: / # Name : AhrefsBot # Info : http://ahrefs.com/robot/ # Header: Mozilla/5.0 (compatible; AhrefsBot/5.1; +http://ahrefs.com/robot/) User-agent: AhrefsBot Disallow: / # Name : MegaIndex.ru # Info : http://megaindex.com/crawler # Header: Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler) User-agent: MegaIndex.ru Disallow: / # Name : aiHitBot # Info : https://www.aihitdata.com/about # Header: Mozilla/5.0 (compatible; aiHitBot/2.9; +https://www.aihitdata.com/about) User-agent: aiHitBot Disallow: / # Name : BLEXBot # Info : http://webmeup-crawler.com/ # Header: Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/) User-agent: BLEXBot Disallow: / # Name : BOT # Info : # Header: BOT/0.1 (BOT for JCE) User-agent: BOT Disallow: / # Name : CCBot # Info : http://commoncrawl.org/faq/ # Header: CCBot/2.0 (http://commoncrawl.org/faq/) User-agent: CCBot Disallow: / # Name : CheckMarkNetwork # Info : http://www.checkmarknetwork.com/spider.html # Header: CheckMarkNetwork/1.0 (+http://www.checkmarknetwork.com/spider.html) User-agent: CheckMarkNetwork Disallow: / # Name : Cliqzbot # Info : http://cliqz.com/company/cliqzbot # Header: Mozilla/5.0 (compatible; Cliqzbot/1.0 +http://cliqz.com/company/cliqzbot) User-agent: Cliqzbot Disallow: / # Name : coccocbot-web # Info : http://help.coccoc.com/searchengine # Header: Mozilla/5.0 (compatible; coccocbot-web/1.0; +http://help.coccoc.com/searchengine) User-agent: coccocbot-web Disallow: / # Name : DeuSu # Info : https://deusu.de/robot.html # Header: Mozilla/5.0 (compatible; DeuSu/5.0.2; +https://deusu.de/robot.html) User-agent: DeuSu Disallow: / # Name : DomainStatsBot # Info : http://domainstats.io/our-bot # Header: DomainStatsBot/1.0 (http://domainstats.io/our-bot) User-agent: DomainStatsBot Disallow: / # Name : DotBot # Info : http://www.opensiteexplorer.org/dotbot, # Header: Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com) User-agent: DotBot Disallow: / # Name : eCairn-Grabber # Info : http://ecairn.com/grabber # Header: Mozilla/5.0 eCairn-Grabber/1.0 (+http://ecairn.com/grabber) User-agent: eCairn-Grabber Disallow: / # Name : Exabot # Info : http://www.exabot.com/go/robot # Header: Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) User-agent: Exabot Disallow: / # Name : FatBot # Info : http://www.thefind.com/crawler # Header: Mozilla/5.0 (compatible; FatBot 2.0; http://www.thefind.com/crawler) User-agent: FatBot Disallow: / # Name : GrapeshotCrawler # Info : http://www.grapeshot.co.uk/crawler.php # Header: Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php) User-agent: GrapeshotCrawler Disallow: / # Name : HaosouSpider # Info : http://www.haosou.com/help/help_3_2.html # Header: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html) User-agent: HaosouSpider Disallow: / # Name : linkdexbot # Info : http://www.linkdex.com/bots/ # Header: Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/bots/) User-agent: linkdexbot Disallow: / # Name : ltx71 # Info : http://ltx71.com/ # Header: ltx71 - (http://ltx71.com/) User-agent: ltx71 Disallow: / # Name : Mail.RU_Bot # Info : http://go.mail.ru/help/robots # Header: Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots) User-agent: Mail.RU_Bot Disallow: / # Name : memoryBot # Info : http://mignify.com/bot.html # Header: Mozilla/5.0 (compatible; memoryBot/1.21.14 +http://mignify.com/bot.html) User-agent: memoryBot Disallow: / # Name : psbot # Info : http://www.picsearch.com/bot.html # Header: psbot/0.1 (+http://www.picsearch.com/bot.html) User-agent: psbot Disallow: / # Name : roboto # Info : # Header: roboto User-agent: roboto Disallow: / # Name : SecurityResearch.bot # Info : http://besome1.info/securityresearchbot.html # Header: Mozilla/5.0 (compatible; SecurityResearch.bot; +http://besome1.info/securityresearchbot.html) User-agent: SecurityResearch.bot Disallow: / # Name : SEOkicks-Robot # Info : http://www.seokicks.de/robot.html # Header: Mozilla/5.0 (compatible; SEOkicks-Robot; +http://www.seokicks.de/robot.html) User-agent: SEOkicks-Robot Disallow: / # Name : SurdotlyBot # Info : http://sur.ly/bot.html # Header: Mozilla/5.0 (compatible; SurdotlyBot/1.0; +http://sur.ly/bot.html) User-agent: SurdotlyBot Disallow: / # Name : WBSearchBot # Info : http://www.warebay.com/bot.html # Header: Mozilla/5.0 (compatible; WBSearchBot/1.1; +http://www.warebay.com/bot.html) User-agent: WBSearchBot Disallow: / # Name : Wotbox # Info : http://www.wotbox.com/bot/ # Header: Wotbox/2.01 (+http://www.wotbox.com/bot/) User-agent: Wotbox Disallow: / # Name : yoozBot-2.2 # Info : http://yooz.ir # Header: Mozilla/5.0 (compatible; yoozBot-2.2; http://yooz.ir; info@yooz.ir) User-agent: yoozBot-2.2 Disallow: / User-agent: Bingbot Crawl-delay: 60 Disallow: User-agent: Googlebot Crawl-delay: 60 Disallow: User-agent: Pinterest Crawl-delay: 60 Disallow: User-agent: Pinterestbot Crawl-delay: 60 Disallow: User-agent: Applebot Crawl-delay: 60 Disallow: User-agent: facebookcatalog/1.0 Crawl-delay: 60 Disallow: # Private pages Disallow: /*?order= Disallow: /*?tag= Disallow: /*?id_currency= Disallow: /*?search_query= Disallow: /*?back= Disallow: /*?n= Disallow: /*&order= Disallow: /*&tag= Disallow: /*&id_currency= Disallow: /*&search_query= Disallow: /*&back= Disallow: /*&n= Disallow: /*controller=addresses Disallow: /*controller=address Disallow: /*controller=authentication Disallow: /*controller=cart Disallow: /*controller=discount Disallow: /*controller=footer Disallow: /*controller=get-file Disallow: /*controller=header Disallow: /*controller=history Disallow: /*controller=identity Disallow: /*controller=images.inc Disallow: /*controller=init Disallow: /*controller=my-account Disallow: /*controller=order Disallow: /*controller=order-slip Disallow: /*controller=order-detail Disallow: /*controller=order-follow Disallow: /*controller=order-return Disallow: /*controller=order-confirmation Disallow: /*controller=pagination Disallow: /*controller=password Disallow: /*controller=pdf-invoice Disallow: /*controller=pdf-order-return Disallow: /*controller=pdf-order-slip Disallow: /*controller=product-sort Disallow: /*controller=search Disallow: /*controller=statistics Disallow: /*controller=attachment Disallow: /*controller=guest-tracking # Directories for dev.trendy-vlasy.cz Disallow: /app/ Disallow: /cache/ Disallow: /classes/ Disallow: /config/ Disallow: /controllers/ Disallow: /download/ Disallow: /js/ Disallow: /localization/ Disallow: /log/ Disallow: /mails/ Disallow: /modules/ Disallow: /override/ Disallow: /pdf/ Disallow: /src/ Disallow: /tools/ Disallow: /translations/ Disallow: /upload/ Disallow: /var/ Disallow: /vendor/ Disallow: /webservice/ Disallow: /cs/app/ Disallow: /cs/cache/ Disallow: /cs/classes/ Disallow: /cs/config/ Disallow: /cs/controllers/ Disallow: /cs/download/ Disallow: /cs/js/ Disallow: /cs/localization/ Disallow: /cs/log/ Disallow: /cs/mails/ Disallow: /cs/modules/ Disallow: /cs/override/ Disallow: /cs/pdf/ Disallow: /cs/src/ Disallow: /cs/tools/ Disallow: /cs/translations/ Disallow: /cs/upload/ Disallow: /cs/var/ Disallow: /cs/vendor/ Disallow: /cs/webservice/ Disallow: /sk/app/ Disallow: /sk/cache/ Disallow: /sk/classes/ Disallow: /sk/config/ Disallow: /sk/controllers/ Disallow: /sk/download/ Disallow: /sk/js/ Disallow: /sk/localization/ Disallow: /sk/log/ Disallow: /sk/mails/ Disallow: /sk/modules/ Disallow: /sk/override/ Disallow: /sk/pdf/ Disallow: /sk/src/ Disallow: /sk/tools/ Disallow: /sk/translations/ Disallow: /sk/upload/ Disallow: /sk/var/ Disallow: /sk/vendor/ Disallow: /sk/webservice/ # Directories for dev.trendy-vlasy.sk Disallow: /app/ Disallow: /cache/ Disallow: /classes/ Disallow: /config/ Disallow: /controllers/ Disallow: /download/ Disallow: /js/ Disallow: /localization/ Disallow: /log/ Disallow: /mails/ Disallow: /modules/ Disallow: /override/ Disallow: /pdf/ Disallow: /src/ Disallow: /tools/ Disallow: /translations/ Disallow: /upload/ Disallow: /var/ Disallow: /vendor/ Disallow: /webservice/ Disallow: /cs/app/ Disallow: /cs/cache/ Disallow: /cs/classes/ Disallow: /cs/config/ Disallow: /cs/controllers/ Disallow: /cs/download/ Disallow: /cs/js/ Disallow: /cs/localization/ Disallow: /cs/log/ Disallow: /cs/mails/ Disallow: /cs/modules/ Disallow: /cs/override/ Disallow: /cs/pdf/ Disallow: /cs/src/ Disallow: /cs/tools/ Disallow: /cs/translations/ Disallow: /cs/upload/ Disallow: /cs/var/ Disallow: /cs/vendor/ Disallow: /cs/webservice/ Disallow: /sk/app/ Disallow: /sk/cache/ Disallow: /sk/classes/ Disallow: /sk/config/ Disallow: /sk/controllers/ Disallow: /sk/download/ Disallow: /sk/js/ Disallow: /sk/localization/ Disallow: /sk/log/ Disallow: /sk/mails/ Disallow: /sk/modules/ Disallow: /sk/override/ Disallow: /sk/pdf/ Disallow: /sk/src/ Disallow: /sk/tools/ Disallow: /sk/translations/ Disallow: /sk/upload/ Disallow: /sk/var/ Disallow: /sk/vendor/ Disallow: /sk/webservice/ # Files Disallow: /*cs/adresa Disallow: /*cs/adresy Disallow: /*cs/prihlasit-se Disallow: /*cs/kosik Disallow: /*cs/sleva Disallow: /*cs/sledovani-objednavky-hoste Disallow: /*cs/historie-objednavek Disallow: /*cs/osobni-udaje Disallow: /*cs/muj-ucet Disallow: /*cs/objednavka Disallow: /*cs/potvrzeni-objednavky Disallow: /*cs/sledovani-objednavky Disallow: /*cs/dobropis Disallow: /*cs/obnova-hesla Disallow: /*cs/vyhledavani Disallow: /*sk/adresa Disallow: /*sk/adresy Disallow: /*sk/prihlasit-sa Disallow: /*sk/kosik Disallow: /*sk/zlavove-kupony Disallow: /*sk/sledovanie-objednavok-pre-navstevnikov Disallow: /*sk/historia-objednavok Disallow: /*sk/osobne-udaje Disallow: /*sk/moj-ucet Disallow: /*sk/objednavka Disallow: /*sk/potvrdenie-objednavky Disallow: /*sk/sledovanie-objednavky Disallow: /*sk/dobropisy Disallow: /*sk/obnova-hesla Disallow: /*sk/vyhladavanie