The Technical AI-Readiness Checklist for Your Site

Which crawlers are allowed?

Run 11 pass-or-fail checks before anything else: four confirm AI crawlers can reach your site at all, seven confirm your homepage has the structural basics engines use once they do reach it. None of this guarantees a citation — it's the floor a brand needs cleared before content and PR work has anywhere to land. See is your site invisible to AI crawlers for what happens when it isn't, and a full robots.txt guide for exact syntax.

Blocking one bot doesn't block the others — blocking GPTBot specifically does not remove you from ChatGPT's answers, a common and costly mistake covered in full in blocking GPTBot isn't blocking ChatGPT.

Which structural elements are present?

Which checks matter most, and how to prioritise failures?

Crawler access first, always. If a crawler can't fetch your site, nothing else on this list matters for that engine — there's no later step where a blocked crawler reconsiders. Extractable body text and a clear H1 next: a page a crawler can't read has nothing for a model to extract, regardless of its schema or meta tags. The rest (meta description, canonical, JSON-LD, last-modified) are low-effort, independent fixes — any order is fine. llms.txt last, deliberately: Google's generative AI guide states you "don't need to create new machine readable files, AI text files, markup, or Markdown" and that doing so "won't help or harm." It's one check on this list, not a priority — see whether to bother with llms.txt for the full evidence picture.

A worked example

A fictional small SaaS homepage, checked today — 7 of 11 passed (64%):

Priority order: fix #2 (crawler access) first, then #7 (quick win), then #5 and #11 as lower-priority cleanup.

Common mistakes: blocking GPTBot and assuming it removes you from ChatGPT entirely; prioritising llms.txt ahead of basic crawler access; never re-checking robots.txt after a CMS or hosting migration, which commonly regenerates an overly-restrictive default.

Checklist:

• Checked /robots.txt for GPTBot, PerplexityBot, Google-Extended, ClaudeBot — including under a wildcard User-agent: * block.
• Loaded the homepage with JavaScript disabled and confirmed the H1 and body text are still there.
• Checked page source for application/ld+json, a meta description, and a canonical tag.
• Checked response headers for Last-Modified.
• Checked whether /llms.txt resolves — optional, not a priority.

Tools to use: the AI Bot Access Checker reads your live robots.txt and reports pass/fail for all four crawlers in seconds; a browser's "disable JavaScript" toggle for the text-extraction check; curl -I on your homepage for response headers.

How to re-check after fixes?

Re-run the full 11-check list after any fix — a robots.txt edit is easy to get subtly wrong. Re-check again after any CMS migration or hosting change, since these commonly reset robots.txt to a restrictive default without anyone noticing until citations drop.

FAQ

Is this the same 11 checks as ContextOwl's paid report? Yes — the free checklist here and ContextOwl's Technical AI-Readiness report chapter run the same 11 checks.

Do I need to pass all 11 to be "AI-ready"? No single threshold is meaningful. Treat crawler access and extractable content as closer to required, and the rest as good practice.

Should I prioritise llms.txt? No. Google's own guide says it neither helps nor hurts in Google Search, and it's the lowest-priority item here.

How often should I re-run this checklist? Immediately after any migration, CMS change or robots.txt edit; quarterly otherwise.

What to do next

1. Run the AI Bot Access Checker now to clear the four crawler-access checks in one pass.
2. Walk the seven structural checks using the checklist above.
3. Fix failures in priority order and re-run the full list after each fix.

ContextOwl sells one-time AI visibility reports. The AI Bot Access Checker is a free version of four of the 11 checks in ContextOwl's full technical-readiness audit.