Websites that block Googlebot
1,014 of the 67,673 readable sites in the census (1.5%) disallow Googlebot (Google, classic-search) from the site root in robots.txt. Classic search crawlers, shown as the baseline.
| Site | Popularity | robots.txt | Read on |
|---|---|---|---|
| api.whatsapp.com | Top 1,000 | 22 disallowed | 2026-10-13 |
| reddit.com | Top 1,000 | 22 disallowed | 2026-10-13 |
| wpa.qq.com | Top 1,000 | 22 disallowed | 2026-10-13 |
| mailchi.mp | Top 1,000 | 22 disallowed | 2026-10-13 |
| chat.whatsapp.com | Top 1,000 | 22 disallowed | 2026-10-13 |
| openssl.org | Top 1,000 | 22 disallowed | 2026-10-13 |
| co.com | Top 1,000 | 22 disallowed | 2026-10-13 |
| qcloud.com | Top 1,000 | 22 disallowed | 2026-10-13 |
| urldefense.com | Top 1,000 | 22 disallowed | 2026-10-13 |
| 1drv.ms | Top 1,000 | 22 disallowed | 2026-10-13 |
| wa.gov | Top 1,000 | 22 disallowed | 2026-10-13 |
| fbcdn.net | Top 1,000 | 22 disallowed | 2026-10-13 |
| aka.ms | Top 1,000 | 22 disallowed | 2026-10-13 |
| xiaohongshu.com | Top 1,000 | 21 disallowed | 2026-10-13 |
| pantheonsite.io | Top 10,000 | 22 disallowed | 2026-10-13 |
| rs6.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| oxfordjournals.org | Top 10,000 | 22 disallowed | 2026-10-13 |
| gstatic.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| arte.tv | Top 10,000 | 22 disallowed | 2026-10-13 |
| helsinki.fi | Top 10,000 | 22 disallowed | 2026-10-13 |
| mcusercontent.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| click.linksynergy.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| cnki.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| hostingersite.com | Top 10,000 | 1 disallowed | 2026-10-13 |
| home.blog | Top 10,000 | 22 disallowed | 2026-10-13 |
| wordstream.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| lexisnexis.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| weibo.cn | Top 10,000 | 22 disallowed | 2026-10-13 |
| anrdoezrs.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| pxf.io | Top 10,000 | 22 disallowed | 2026-10-13 |
| sjv.io | Top 10,000 | 22 disallowed | 2026-10-13 |
| connect.ok.ru | Top 10,000 | 22 disallowed | 2026-10-13 |
| bangkokpost.com | Top 10,000 | 2 disallowed | 2026-10-13 |
| jdoqocy.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| dpbolvw.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| tkqlhce.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| kqzyfj.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| ad.doubleclick.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| england.nhs.uk | Top 10,000 | 22 disallowed | 2026-10-13 |
| nv.gov | Top 10,000 | 22 disallowed | 2026-10-13 |
| 1wcasino.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| secureserver.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| magento.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| mjt.lu | Top 10,000 | 22 disallowed | 2026-10-13 |
| page-stats.de | Top 10,000 | 2 disallowed | 2026-10-13 |
| toronto.edu | Top 10,000 | 22 disallowed | 2026-10-13 |
| pbc.gov.cn | Top 10,000 | 22 disallowed | 2026-10-13 |
| overdrive.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| healthdirect.gov.au | Top 10,000 | 22 disallowed | 2026-10-13 |
| asahi-net.or.jp | Top 10,000 | 22 disallowed | 2026-10-13 |
| hubs.ly | Top 10,000 | 22 disallowed | 2026-10-13 |
| casinonsvenska.eu | Top 10,000 | 22 disallowed | 2026-10-13 |
| hs-sites.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| jiangsu.gov.cn | Top 10,000 | 3 disallowed | 2026-10-13 |
| pr.gov.br | Top 10,000 | 22 disallowed | 2026-10-13 |
| akkadia.org | Top 10,000 | 22 disallowed | 2026-10-13 |
| arcor.de | Top 10,000 | 22 disallowed | 2026-10-13 |
| qrco.de | Top 10,000 | 22 disallowed | 2026-10-13 |
| depositfiles.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| catholicnewsagency.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| ebird.org | Top 10,000 | 22 disallowed | 2026-10-13 |
| asiaone.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| moa.gov.cn | Top 10,000 | 22 disallowed | 2026-10-13 |
| cmail20.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| cmail19.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| xanga.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| shapeways.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| litmus.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| redirectingat.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| law-gov.cn | Top 10,000 | 22 disallowed | 2026-10-13 |
| skimresources.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| dzszyjsxx.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| megaphone.fm | Top 10,000 | 22 disallowed | 2026-10-13 |
| startersites.io | Top 10,000 | 10 disallowed | 2026-10-13 |
| widen.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| href.li | Top 10,000 | 22 disallowed | 2026-10-13 |
| missingkids.org | Top 10,000 | 22 disallowed | 2026-10-13 |
| china-heating.org.cn | Top 10,000 | 22 disallowed | 2026-10-13 |
| nccu.edu.tw | Top 10,000 | 22 disallowed | 2026-10-13 |
| qbnz.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| campaign.gov.uk | Top 10,000 | 22 disallowed | 2026-10-13 |
| sendibm3.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| sp1-brevo.net | Top 10,000 | 22 disallowed | 2026-10-13 |
| yidianzixun.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| createsend1.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| nzn-oa.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| backblazeb2.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| treasurydirect.gov | Top 10,000 | 22 disallowed | 2026-10-13 |
| jotformeu.com | Top 10,000 | 22 disallowed | 2026-10-13 |
| judiciary.uk | Top 10,000 | 22 disallowed | 2026-10-13 |
| cryptome.org | Top 50,000 | 2 disallowed | 2026-10-13 |
| derstandard.de | Top 50,000 | 22 disallowed | 2026-10-13 |
| byt360.com | Top 50,000 | 2 disallowed | 2026-10-13 |
| share.transistor.fm | Top 50,000 | 1 disallowed | 2026-10-13 |
| perma.cc | Top 50,000 | 1 disallowed | 2026-10-13 |
| rstyle.me | Top 50,000 | 20 disallowed | 2026-10-13 |
| ly.com | Top 50,000 | 1 disallowed | 2026-10-13 |
| lofter.com | Top 50,000 | 1 disallowed | 2026-10-13 |
| lanzoui.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| planning.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| elsevierpure.com | Top 50,000 | 21 disallowed | 2026-10-13 |
| brighteon.com | Top 50,000 | 1 disallowed | 2026-10-13 |
| easy.co | Top 50,000 | 22 disallowed | 2026-10-13 |
| yangkeduo.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| cnnic.cn | Top 50,000 | 2 disallowed | 2026-10-13 |
| jhtongchuang.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| cnnic.net.cn | Top 50,000 | 2 disallowed | 2026-10-13 |
| lanzouw.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzouu.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| antipope.org | Top 50,000 | 11 disallowed | 2026-10-13 |
| scontent.xx.fbcdn.net | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzout.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| jobkorea.co.kr | Top 50,000 | 22 disallowed | 2026-10-13 |
| zx3315.cn | Top 50,000 | 22 disallowed | 2026-10-13 |
| kbstar.com | Top 50,000 | 21 disallowed | 2026-10-13 |
| fedorapeople.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| startups.co.uk | Top 50,000 | 19 disallowed | 2026-10-13 |
| gnome-look.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| jornaldenegocios.pt | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzoul.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzoub.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| wilmerhale.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| yelp.co.uk | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzoue.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzouq.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| hihostels.com | Top 50,000 | 14 disallowed | 2026-10-13 |
| lanzoum.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| cat-v.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| rockbox.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| rpb-15.ru | Top 50,000 | 1 disallowed | 2026-10-13 |
| stargard.pl | Top 50,000 | 2 disallowed | 2026-10-13 |
| macaulaylibrary.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| pling.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzouv.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| stcilisyxz.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| qvigrassupport.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| yelp.de | Top 50,000 | 22 disallowed | 2026-10-13 |
| hpkucunmaoyi.com | Top 50,000 | 1 disallowed | 2026-10-13 |
| ha.org.hk | Top 50,000 | 22 disallowed | 2026-10-13 |
| sharethrough.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| opendesktop.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| canva.cn | Top 50,000 | 10 disallowed | 2026-10-13 |
| inube.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzoux.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| tradepub.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| roodo.com | Top 50,000 | 1 disallowed | 2026-10-13 |
| dongchedi.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| factmags.com | Top 50,000 | 1 disallowed | 2026-10-13 |
| pinupcasinokz.kz | Top 50,000 | 22 disallowed | 2026-10-13 |
| irdai.gov.in | Top 50,000 | 22 disallowed | 2026-10-13 |
| jotform.me | Top 50,000 | 22 disallowed | 2026-10-13 |
| lanzouo.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| userstyles.world | Top 50,000 | 22 disallowed | 2026-10-13 |
| thebell.co.kr | Top 50,000 | 22 disallowed | 2026-10-13 |
| goooogla.com | Top 50,000 | 1 disallowed | 2026-10-13 |
| info-retraite.fr | Top 50,000 | 22 disallowed | 2026-10-13 |
| ebay.io | Top 50,000 | 22 disallowed | 2026-10-13 |
| flipboard.social | Top 50,000 | 11 disallowed | 2026-10-13 |
| buzzflash.com | Top 50,000 | 22 disallowed | 2026-10-13 |
| kde-look.org | Top 50,000 | 22 disallowed | 2026-10-13 |
| cehong.com.cn | Top 100,000 | 22 disallowed | 2026-10-13 |
| pin-upkazakhstan.com | Top 100,000 | 22 disallowed | 2026-10-13 |
| blaw.org.cn | Top 100,000 | 22 disallowed | 2026-10-13 |
| dgs.pt | Top 100,000 | 2 disallowed | 2026-10-13 |
| gigapan.com | Top 100,000 | 22 disallowed | 2026-10-13 |
| cisd.com.cn | Top 100,000 | 1 disallowed | 2026-10-13 |
| jotform.us | Top 100,000 | 22 disallowed | 2026-10-13 |
| worldtaekwondo.org | Top 100,000 | 21 disallowed | 2026-10-13 |
| newsmth.net | Top 100,000 | 22 disallowed | 2026-10-13 |
| zefix.ch | Top 100,000 | 22 disallowed | 2026-10-13 |
| magiran.com | Top 100,000 | 20 disallowed | 2026-10-13 |
| lun.com | Top 100,000 | 1 disallowed | 2026-10-13 |
| officeplus.cn | Top 100,000 | 1 disallowed | 2026-10-13 |
| quicket.co.za | Top 100,000 | 22 disallowed | 2026-10-13 |
| cqhot.cn | Top 100,000 | 21 disallowed | 2026-10-13 |
| jotform.co | Top 100,000 | 22 disallowed | 2026-10-13 |
| upmedia.mg | Top 100,000 | 22 disallowed | 2026-10-13 |
| jetsetter.com | Top 100,000 | 22 disallowed | 2026-10-13 |
The list shows the sites that have a page here, most popular first. This reports what the file asks. It does not show whether a crawler can actually reach the site: many sites also block crawlers at their CDN or firewall, and some crawlers ignore robots.txt.