Last updated
Should you block AI crawlers from your website?
For a business that wants to be found, generally no. Blocking removes the site from the pool assistants draw on when recommending someone, which costs visibility. Blocking makes sense for publishers whose product is the writing itself, and for anyone whose content is licensed rather than promotional.
The case for blocking
It is real for publishers. If the writing is the product, having it absorbed and summarised without a click is a direct loss, and blocking is a reasonable negotiating position while licensing arrangements are still being worked out. That logic does not transfer to a roofing company whose website exists to produce phone calls.
The case against, for everyone else
A local business is trying to be mentioned. Being absent from the sources an assistant consults means being absent from the answer, and the competitor who stayed open gets named instead. There is no partial credit for having been protective.
The middle position
Allow the search and retrieval crawlers, since those are what put a business into answers. Decide separately about the training crawlers, where the tradeoff is genuinely arguable. Keep anything confidential behind a login, because that is the only control that actually enforces itself.
Start with the free audit.
Get a free auditSend the link. We read the site by hand and write back inside five business days.
No call, no commitment