# robots.txt for https://economiapertutti.bancaditalia.it # Following rules apply to all public search engines subscribing to # standard http://www.robotstxt.org/orig.html # advertising-related bots: User-agent: Mediapartners-Google Disallow: / # Some bots are known to be trouble: User-agent: BLP_bbot/0.1 Disallow: / User-agent: BLP_bbot Disallow: / User-agent: ^Java Disallow: / User-agent: libwww Disallow: / User-agent: wget Disallow: / User-agent: SiteimproveBot-Crawler Allow: / # Friendly, low-speed bots are welcome viewing article pages, but not # dynamically-generated pages please. User-agent: * Crawl-delay: 5 Disallow: /notizie/ricerca/ Disallow: /ricerca/