This week's changes

2,417 sites changed their robots.txt in the last 7 days.

• pinterest.com (Top 1,000), 2026-10-12: no new disallows
• wordpress.com (Top 1,000), 2026-10-13: no new disallows
• wiley.com (Top 1,000), 2026-10-12: no new disallows
• wp.com (Top 1,000), 2026-10-13: no new disallows
• ovhcloud.com (Top 1,000), 2026-10-12: no new disallows
• tencent.com (Top 1,000), 2026-10-12: no new disallows
• foxnews.com (Top 1,000), 2026-10-13: no new disallows
• dw.com (Top 1,000), 2026-10-13: no new disallows; no longer disallows Perplexity-User
• walmart.com (Top 1,000), 2026-10-12: no new disallows
• spiegel.de (Top 1,000), 2026-10-13: no new disallows; no longer disallows Amazonbot
• generatepress.com (Top 1,000), 2026-10-12: no new disallows
• census.gov (Top 1,000), 2026-10-13: no new disallows
• opensource.org (Top 1,000), 2026-10-12: no new disallows
• meta.com (Top 1,000), 2026-10-13: no new disallows
• pcmag.com (Top 1,000), 2026-10-13: no new disallows
• venturebeat.com (Top 1,000), 2026-10-12: no new disallows
• msu.edu (Top 1,000), 2026-10-13: no new disallows
• virginia.edu (Top 1,000), 2026-10-12: no new disallows
• figma.com (Top 1,000), 2026-10-13: no new disallows
• gumroad.com (Top 1,000), 2026-10-12: no new disallows
• amnesty.org (Top 1,000), 2026-10-13: now disallows CCBot, PerplexityBot, Bingbot
• brandbucket.com (Top 1,000), 2026-10-12: no new disallows
• heylink.me (Top 10,000), 2026-10-12: no new disallows
• thestar.com (Top 10,000), 2026-10-13: no new disallows
• arduino.cc (Top 10,000), 2026-10-12: no new disallows
• all.accor.com (Top 10,000), 2026-10-12: no new disallows
• straitstimes.com (Top 10,000), 2026-10-12: no new disallows
• iu.edu (Top 10,000), 2026-10-12: no new disallows
• leparisien.fr (Top 10,000), 2026-10-13: no new disallows
• wto.org (Top 10,000), 2026-10-12: no new disallows
• lavanguardia.com (Top 10,000), 2026-10-13: no new disallows
• everydayhealth.com (Top 10,000), 2026-10-13: no new disallows
• pinimg.com (Top 10,000), 2026-10-12: no new disallows
• kommersant.ru (Top 10,000), 2026-10-13: no new disallows
• softpedia.com (Top 10,000), 2026-10-12: no new disallows
• hostnet.nl (Top 10,000), 2026-10-12: no new disallows
• bundesregierung.de (Top 10,000), 2026-10-12: no new disallows
• csiro.au (Top 10,000), 2026-10-13: no new disallows
• uni-bonn.de (Top 10,000), 2026-10-13: no new disallows
• wordstream.com (Top 10,000), 2026-10-13: now disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, OAI-SearchBot, PerplexityBot, Claude-SearchBot, DuckAssistBot, ChatGPT-User, Perplexity-User, Claude-User, meta-externalfetcher, Googlebot, Bingbot
• dreamstime.com (Top 10,000), 2026-10-13: no new disallows
• pinterest.de (Top 10,000), 2026-10-12: no new disallows
• towardsdatascience.com (Top 10,000), 2026-10-13: no new disallows
• de.pinterest.com (Top 10,000), 2026-10-12: no new disallows
• kompas.com (Top 10,000), 2026-10-12: no new disallows
• wfp.org (Top 10,000), 2026-10-13: no new disallows
• elevenlabs.io (Top 10,000), 2026-10-12: no new disallows
• informationweek.com (Top 10,000), 2026-10-13: no new disallows
• gamesradar.com (Top 10,000), 2026-10-13: now disallows Applebot-Extended
• designboom.com (Top 10,000), 2026-10-12: no new disallows
• livestrong.com (Top 10,000), 2026-10-13: no new disallows
• uschamber.com (Top 10,000), 2026-10-13: no new disallows
• pinterest.fr (Top 10,000), 2026-10-12: no new disallows
• bangkokpost.com (Top 10,000), 2026-10-13: now disallows Googlebot, Bingbot
• beehiiv.com (Top 10,000), 2026-10-12: no new disallows
• acog.org (Top 10,000), 2026-10-13: no new disallows; no longer disallows GPTBot, Google-Extended, CCBot, omgili, ChatGPT-User
• aappublications.org (Top 10,000), 2026-10-13: no new disallows
• inaturalist.org (Top 10,000), 2026-10-13: now disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili
• kayak.com (Top 10,000), 2026-10-13: no new disallows
• harpercollins.com (Top 10,000), 2026-10-13: no new disallows
• suno.com (Top 10,000), 2026-10-13: no new disallows
• utm.edu (Top 10,000), 2026-10-13: no new disallows
• pinterest.ca (Top 10,000), 2026-10-12: no new disallows
• nola.com (Top 10,000), 2026-10-13: no new disallows
• kia.com (Top 10,000), 2026-10-12: no new disallows
• tau.ac.il (Top 10,000), 2026-10-12: no new disallows
• unimi.it (Top 10,000), 2026-10-12: no new disallows
• tinypng.com (Top 10,000), 2026-10-13: no new disallows
• chess.com (Top 10,000), 2026-10-13: no new disallows
• gamedeveloper.com (Top 10,000), 2026-10-13: no new disallows
• diabetesjournals.org (Top 10,000), 2026-10-13: no new disallows
• flagcounter.com (Top 10,000), 2026-10-12: no new disallows
• contentmarketinginstitute.com (Top 10,000), 2026-10-13: no new disallows
• blossomthemes.com (Top 10,000), 2026-10-13: no new disallows
• aacrjournals.org (Top 10,000), 2026-10-13: no new disallows
• pinterest.es (Top 10,000), 2026-10-12: no new disallows
• stltoday.com (Top 10,000), 2026-10-13: no new disallows
• kraken.com (Top 10,000), 2026-10-12: no new disallows
• accorhotels.com (Top 10,000), 2026-10-12: no new disallows
• kbb.com (Top 10,000), 2026-10-12: no new disallows
• flightaware.com (Top 10,000), 2026-10-13: no new disallows
• broadwayworld.com (Top 10,000), 2026-10-13: no new disallows
• synthesia.io (Top 10,000), 2026-10-12: no new disallows
• darkreading.com (Top 10,000), 2026-10-13: no new disallows
• make.com (Top 10,000), 2026-10-12: no new disallows
• cleantechnica.com (Top 10,000), 2026-10-13: no new disallows
• us.es (Top 10,000), 2026-10-12: no new disallows; no longer disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, OAI-SearchBot, PerplexityBot, Claude-SearchBot, DuckAssistBot, ChatGPT-User, Perplexity-User, Claude-User, meta-externalfetcher, Googlebot, Bingbot
• wpmet.com (Top 10,000), 2026-10-12: no new disallows
• mediapart.fr (Top 10,000), 2026-10-13: no new disallows
• theprint.in (Top 10,000), 2026-10-13: no new disallows; no longer disallows GPTBot
• eluniversal.com.mx (Top 10,000), 2026-10-13: no new disallows
• timesnownews.com (Top 10,000), 2026-10-13: no new disallows
• diepresse.com (Top 10,000), 2026-10-13: no new disallows; no longer disallows Google-Extended
• dejure.org (Top 10,000), 2026-10-12: no new disallows
• london.gov.uk (Top 10,000), 2026-10-13: no new disallows
• sika.com (Top 10,000), 2026-10-13: no new disallows
• backblaze.com (Top 10,000), 2026-10-13: no new disallows
• audacy.com (Top 10,000), 2026-10-12: no new disallows; no longer disallows Bingbot
• cnnbrasil.com.br (Top 10,000), 2026-10-13: no new disallows
• atom.com (Top 10,000), 2026-10-12: no new disallows
• ixbt.com (Top 10,000), 2026-10-12: no new disallows
• healthdirect.gov.au (Top 10,000), 2026-10-13: now disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, OAI-SearchBot, PerplexityBot, Claude-SearchBot, DuckAssistBot, ChatGPT-User, Perplexity-User, Claude-User, meta-externalfetcher, Googlebot, Bingbot
• gamasutra.com (Top 10,000), 2026-10-13: no new disallows
• unito.it (Top 10,000), 2026-10-12: no new disallows
• preply.com (Top 10,000), 2026-10-13: no new disallows
• alternativeto.net (Top 10,000), 2026-10-12: no new disallows
• swarthmore.edu (Top 10,000), 2026-10-13: no new disallows
• mapy.cz (Top 10,000), 2026-10-12: no new disallows
• buffalonews.com (Top 10,000), 2026-10-13: no new disallows
• mundodeportivo.com (Top 10,000), 2026-10-12: no new disallows
• lichess.org (Top 10,000), 2026-10-12: no new disallows
• boosty.to (Top 10,000), 2026-10-12: no new disallows
• securelist.com (Top 10,000), 2026-10-13: no new disallows
• skf.com (Top 10,000), 2026-10-13: no new disallows
• tucson.com (Top 10,000), 2026-10-12: no new disallows
• brennancenter.org (Top 10,000), 2026-10-12: no new disallows
• sacred-texts.com (Top 10,000), 2026-10-12: no new disallows
• demilked.com (Top 10,000), 2026-10-12: now disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili
• daaz.com (Top 10,000), 2026-10-13: now disallows GPTBot, ClaudeBot, Google-Extended, CCBot, Applebot-Extended
• pasteur.fr (Top 10,000), 2026-10-13: no new disallows
• postandcourier.com (Top 10,000), 2026-10-13: no new disallows
• nationalrail.co.uk (Top 10,000), 2026-10-13: no new disallows
• scimagojr.com (Top 10,000), 2026-10-13: no new disallows
• autoevolution.com (Top 10,000), 2026-10-12: no new disallows
• ivermedx.com (Top 10,000), 2026-10-12: no new disallows
• sdu.dk (Top 10,000), 2026-10-13: no new disallows
• ashrae.org (Top 10,000), 2026-10-13: no new disallows
• qnap.com (Top 10,000), 2026-10-12: no new disallows
• madison.com (Top 10,000), 2026-10-13: no new disallows
• manager-magazin.de (Top 10,000), 2026-10-13: no new disallows; no longer disallows Amazonbot
• pinterest.it (Top 10,000), 2026-10-12: no new disallows
• port.ac.uk (Top 10,000), 2026-10-13: no new disallows
• crave.ca (Top 10,000), 2026-10-13: no new disallows
• letour.fr (Top 10,000), 2026-10-12: now disallows meta-externalagent
• waikato.ac.nz (Top 10,000), 2026-10-13: no new disallows
• fender.com (Top 10,000), 2026-10-13: no new disallows
• tmcnet.com (Top 10,000), 2026-10-13: no new disallows
• federatedjournals.com (Top 10,000), 2026-10-12: no new disallows
• 104.com.tw (Top 10,000), 2026-10-13: no new disallows
• conference-board.org (Top 10,000), 2026-10-12: no new disallows
• sfexaminer.com (Top 10,000), 2026-10-13: no new disallows
• betterhelp.com (Top 10,000), 2026-10-13: no new disallows
• versobooks.com (Top 10,000), 2026-10-13: no new disallows
• carscoops.com (Top 10,000), 2026-10-13: no new disallows
• iccsafe.org (Top 10,000), 2026-10-13: no new disallows
• gigazine.net (Top 10,000), 2026-10-12: no new disallows
• richmond.com (Top 10,000), 2026-10-13: no new disallows
• wcvb.com (Top 10,000), 2026-10-13: no new disallows
• manualslib.com (Top 10,000), 2026-10-13: no new disallows
• istudiosg.com (Top 10,000), 2026-10-13: no new disallows
• omaha.com (Top 10,000), 2026-10-13: no new disallows
• carrier.com (Top 10,000), 2026-10-13: no new disallows
• viamichelin.com (Top 10,000), 2026-10-13: no new disallows
• trendyol.com (Top 10,000), 2026-10-13: no new disallows
• id.pinterest.com (Top 10,000), 2026-10-12: no new disallows
• dfg.de (Top 10,000), 2026-10-13: no new disallows
• paralympic.org (Top 10,000), 2026-10-13: no new disallows
• ratgeberrecht.eu (Top 10,000), 2026-10-13: no new disallows; no longer disallows Google-Extended, Amazonbot, Diffbot, omgili, OAI-SearchBot, PerplexityBot, ChatGPT-User
• laptopmag.com (Top 10,000), 2026-10-13: now disallows Applebot-Extended
• theadvocate.com (Top 10,000), 2026-10-12: no new disallows
• storeboard.com (Top 10,000), 2026-10-12: no new disallows
• lightreading.com (Top 10,000), 2026-10-13: no new disallows
• johnsoncontrols.com (Top 10,000), 2026-10-13: no new disallows
• omnicalculator.com (Top 10,000), 2026-10-13: no new disallows
• jcyl.es (Top 10,000), 2026-10-13: no new disallows
• amoxihub.com (Top 10,000), 2026-10-12: no new disallows
• anker.com (Top 10,000), 2026-10-13: no new disallows
• xnview.com (Top 10,000), 2026-10-12: no new disallows
• androidheadlines.com (Top 10,000), 2026-10-13: no new disallows
• daytondailynews.com (Top 10,000), 2026-10-13: no new disallows
• kcra.com (Top 10,000), 2026-10-13: no new disallows
• glosbe.com (Top 10,000), 2026-10-13: no new disallows
• aoa.org (Top 10,000), 2026-10-12: no new disallows
• khan.co.kr (Top 10,000), 2026-10-12: no new disallows
• ashpublications.org (Top 10,000), 2026-10-13: no new disallows
• harvardmagazine.com (Top 10,000), 2026-10-12: no new disallows
• heinonline.org (Top 10,000), 2026-10-13: no new disallows
• form-mailer.jp (Top 10,000), 2026-10-12: no new disallows
• tulsaworld.com (Top 10,000), 2026-10-13: no new disallows
• pepperdine.edu (Top 10,000), 2026-10-13: no new disallows
• tk.de (Top 10,000), 2026-10-13: no new disallows
• tsmc.com (Top 10,000), 2026-10-12: no new disallows
• digitalcameraworld.com (Top 10,000), 2026-10-13: now disallows Applebot-Extended
• cas.org (Top 10,000), 2026-10-12: no new disallows
• armorgames.com (Top 10,000), 2026-10-13: no new disallows
• fairphone.com (Top 10,000), 2026-10-12: no new disallows
• suzhoudress.com (Top 10,000), 2026-10-13: no new disallows
• pcloud.com (Top 10,000), 2026-10-13: no new disallows
• legal500.com (Top 10,000), 2026-10-13: no new disallows
• firstthings.com (Top 10,000), 2026-10-13: no new disallows
• sfweekly.com (Top 10,000), 2026-10-13: no new disallows
• toptenreviews.com (Top 10,000), 2026-10-13: now disallows Applebot-Extended
• cadence.com (Top 10,000), 2026-10-12: now disallows GPTBot, CCBot; no longer disallows Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, Claude-SearchBot, DuckAssistBot, Perplexity-User, Claude-User, meta-externalfetcher
• bancaditalia.it (Top 10,000), 2026-10-12: no new disallows
• pl.pinterest.com (Top 10,000), 2026-10-12: no new disallows
• laughingsquid.com (Top 10,000), 2026-10-13: no new disallows
• cpubenchmark.net (Top 10,000), 2026-10-12: now disallows ClaudeBot, Google-Extended, CCBot, meta-externalagent, Applebot-Extended, cohere-ai, Diffbot, omgili; no longer disallows Bingbot
• beinsports.com (Top 10,000), 2026-10-13: no new disallows
• mediamarkt.de (Top 10,000), 2026-10-13: no new disallows
• spacedaily.com (Top 10,000), 2026-10-13: no new disallows
• todayonline.com (Top 10,000), 2026-10-13: no new disallows; no longer disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, OAI-SearchBot, PerplexityBot, Claude-SearchBot, DuckAssistBot, ChatGPT-User, Perplexity-User, Claude-User, meta-externalfetcher, Googlebot, Bingbot
• tribuna.com (Top 10,000), 2026-10-13: no new disallows
• whathifi.com (Top 10,000), 2026-10-13: now disallows Applebot-Extended
• datacenterknowledge.com (Top 10,000), 2026-10-13: no new disallows
• frost.com (Top 10,000), 2026-10-13: no new disallows; no longer disallows CCBot
• advfn.com (Top 10,000), 2026-10-13: no new disallows
• northumbria.ac.uk (Top 10,000), 2026-10-13: no new disallows
• dblp.org (Top 10,000), 2026-10-12: no new disallows; no longer disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, OAI-SearchBot, PerplexityBot, Claude-SearchBot, DuckAssistBot, ChatGPT-User, Perplexity-User, Claude-User, meta-externalfetcher, Googlebot, Bingbot
• informationisbeautiful.net (Top 10,000), 2026-10-12: no new disallows
• atari.com (Top 10,000), 2026-10-13: no new disallows
• mcmaster.com (Top 10,000), 2026-10-13: no new disallows
• cnhcjt.com (Top 10,000), 2026-10-12: no new disallows
• fark.com (Top 10,000), 2026-10-12: now disallows Claude-SearchBot
• nightcafe.studio (Top 10,000), 2026-10-13: no new disallows
• fashionista.com (Top 50,000), 2026-10-13: no new disallows; no longer disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, ChatGPT-User, Perplexity-User, Claude-User, meta-externalfetcher
• pinterest.jp (Top 50,000), 2026-10-12: no new disallows
• iitm.ac.in (Top 50,000), 2026-10-13: no new disallows
• desy.de (Top 50,000), 2026-10-13: now disallows GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, OAI-SearchBot, PerplexityBot, Claude-SearchBot, DuckAssistBot, ChatGPT-User, Perplexity-User, Claude-User, meta-externalfetcher
• compass.com (Top 50,000), 2026-10-13: no new disallows
• toysrus.com (Top 50,000), 2026-10-13: no new disallows
• juventus.com (Top 50,000), 2026-10-13: no new disallows
• uswitch.com (Top 50,000), 2026-10-13: no new disallows
• is.fi (Top 50,000), 2026-10-13: no new disallows
• houstonpublicmedia.org (Top 50,000), 2026-10-13: no new disallows
• mako.co.il (Top 50,000), 2026-10-12: no new disallows
• rockefeller.edu (Top 50,000), 2026-10-13: no new disallows
• withings.com (Top 50,000), 2026-10-13: no new disallows
• walla.co.il (Top 50,000), 2026-10-12: no new disallows
• palm.com (Top 50,000), 2026-10-13: no new disallows
• beyondtrust.com (Top 50,000), 2026-10-13: no new disallows; no longer disallows Diffbot
• phaidon.com (Top 50,000), 2026-10-13: no new disallows
• rajasthan.gov.in (Top 50,000), 2026-10-12: no new disallows
• taxpolicycenter.org (Top 50,000), 2026-10-13: no new disallows
• bikemap.net (Top 50,000), 2026-10-13: no new disallows
• journalstar.com (Top 50,000), 2026-10-13: no new disallows
• nordbayern.de (Top 50,000), 2026-10-13: no new disallows; no longer disallows cohere-ai, PerplexityBot, DuckAssistBot, ChatGPT-User, Perplexity-User
• lancasteronline.com (Top 50,000), 2026-10-13: no new disallows
• ird.fr (Top 50,000), 2026-10-13: no new disallows
• santafenewmexican.com (Top 50,000), 2026-10-13: no new disallows
• thespec.com (Top 50,000), 2026-10-13: no new disallows
• kuka.com (Top 50,000), 2026-10-12: no new disallows
• orkut.com (Top 50,000), 2026-10-12: no new disallows
• wbaltv.com (Top 50,000), 2026-10-13: no new disallows
• geoguessr.com (Top 50,000), 2026-10-12: no new disallows
• wesh.com (Top 50,000), 2026-10-13: no new disallows
• maketecheasier.com (Top 50,000), 2026-10-13: no new disallows
• nuxt.com (Top 50,000), 2026-10-12: no new disallows
• wmur.com (Top 50,000), 2026-10-13: no new disallows
• chain.link (Top 50,000), 2026-10-13: no new disallows
• wfmz.com (Top 50,000), 2026-10-13: no new disallows
• docin.com (Top 50,000), 2026-10-12: no new disallows
• nwitimes.com (Top 50,000), 2026-10-13: no new disallows
• citibank.com (Top 50,000), 2026-10-12: no new disallows
• oclean.com (Top 50,000), 2026-10-13: no new disallows
• thegazette.com (Top 50,000), 2026-10-13: no new disallows
• maccosmetics.com (Top 50,000), 2026-10-13: no new disallows
• thebodyshop.com (Top 50,000), 2026-10-13: no new disallows
• thecentersquare.com (Top 50,000), 2026-10-13: no new disallows
• friendlycaptcha.com (Top 50,000), 2026-10-12: no new disallows
• verdi.de (Top 50,000), 2026-10-13: no new disallows
• extension.org (Top 50,000), 2026-10-12: no new disallows
• acaai.org (Top 50,000), 2026-10-13: no new disallows
• yealink.com (Top 50,000), 2026-10-12: no new disallows
• manus.space (Top 50,000), 2026-10-13: no new disallows
• nrn.com (Top 50,000), 2026-10-13: no new disallows
• pinterest.ru (Top 50,000), 2026-10-12: no new disallows
• cumhuriyet.com.tr (Top 50,000), 2026-10-13: now disallows GPTBot, ClaudeBot, CCBot, meta-externalagent, Bytespider, Applebot-Extended, cohere-ai, Diffbot
• barco.com (Top 50,000), 2026-10-13: no new disallows
• order.online (Top 50,000), 2026-10-13: no new disallows
• amgen.com (Top 50,000), 2026-10-13: no new disallows
• tu.berlin (Top 50,000), 2026-10-13: no new disallows
• billingsgazette.com (Top 50,000), 2026-10-13: no new disallows
• aaronsw.com (Top 50,000), 2026-10-12: no new disallows
• geeky-gadgets.com (Top 50,000), 2026-10-13: no new disallows
• archpaper.com (Top 50,000), 2026-10-12: no new disallows
• bma.org.uk (Top 50,000), 2026-10-12: no new disallows
• joblo.com (Top 50,000), 2026-10-13: no new disallows
• caib.es (Top 50,000), 2026-10-13: no new disallows
• blu-ray.com (Top 50,000), 2026-10-13: no new disallows
• farmprogress.com (Top 50,000), 2026-10-13: no new disallows
• dsv.com (Top 50,000), 2026-10-13: no new disallows
• awn.com (Top 50,000), 2026-10-13: no new disallows
• beeline.ru (Top 50,000), 2026-10-13: no new disallows
• worthpoint.com (Top 50,000), 2026-10-12: no new disallows
• roanoke.com (Top 50,000), 2026-10-13: no new disallows
• edn.com (Top 50,000), 2026-10-13: no new disallows
• bworldonline.com (Top 50,000), 2026-10-13: no new disallows
• nowtoronto.com (Top 50,000), 2026-10-13: no new disallows
• thesundaytimes.co.uk (Top 50,000), 2026-10-13: now disallows ClaudeBot, Google-Extended, CCBot, meta-externalagent, Bytespider, Applebot-Extended, Amazonbot, cohere-ai, Diffbot, omgili, Timpibot, PerplexityBot, Claude-SearchBot, DuckAssistBot, Perplexity-User, Claude-User, meta-externalfetcher
• pinco-kz.win (Top 50,000), 2026-10-13: no new disallows
• nu.edu (Top 50,000), 2026-10-13: no new disallows
• blender.community (Top 50,000), 2026-10-13: no new disallows
• helpshift.com (Top 50,000), 2026-10-13: no new disallows
• crocs.com (Top 50,000), 2026-10-13: no new disallows
• sleepdoctor.com (Top 50,000), 2026-10-13: no new disallows
• umami.is (Top 50,000), 2026-10-13: no new disallows
• eccouncil.org (Top 50,000), 2026-10-12: no new disallows
• dailycal.org (Top 50,000), 2026-10-13: no new disallows
• flyfrontier.com (Top 50,000), 2026-10-13: no new disallows
• gettysburg.edu (Top 50,000), 2026-10-13: no new disallows