Skip to content
AI Search Basics

How Claude, Gemini and Copilot Answer Brand Questions

Each engine documents a little about how it finds sources and leaves most of the rest unpublished. Here's exactly what's confirmed, by whom, and what still isn't.

By ContextOwl Research · Published October 11, 2026 · Last updated October 11, 2026

Key takeaways
  • Claude, Gemini and Copilot each publish some detail about sourcing and leave the rest unconfirmed — this post says exactly where that line falls for each.
  • Claude has three documented bots with distinct jobs: ClaudeBot (training), Claude-SearchBot (search quality), and Claude-User (user-triggered fetch).
  • Gemini's "grounding with Google Search" cites sources with structured metadata, but Google hasn't published whether it shares the same crawl or index as AI Overviews and AI Mode.
  • Copilot runs on Microsoft's Bing index — there's no separate "Copilot crawler"; Bingbot's crawl is what feeds it.
  • ContextOwl's own reports cover ChatGPT as the base engine plus Gemini, Perplexity and Claude as add-ons. Copilot is covered here for comparison only — it isn't a ContextOwl report engine.

What does each engine actually publish about how it finds sources?

Unfamiliar term below? The AI search glossary covers "grounding," "crawler" and the rest in one place — or see AI search vs traditional search for the general retrieve-then-generate pipeline these three engines each build on differently. Claude publishes the most structured detail of the three on crawler-level behavior. Anthropic's support documentation names three separate bots and states plainly that all of them "respect 'do not crawl' signals by honoring industry standard directives in robots.txt," plus the Crawl-delay extension for pacing.

Gemini's public documentation is strongest on the citation mechanics, weaker on the retrieval side. Google's developer docs for "grounding with Google Search" state that when grounding is enabled, "the model handles the entire workflow of searching, processing, and citing information automatically," and that responses carry structured groundingChunks (the actual source URLs and titles) and groundingSupports (which link specific stretches of the answer's text back to those sources) — "the key to building inline citations," in Google's own words. What that page does not say, as of this writing, is whether this grounding process draws on the same crawl and index that power AI Overviews and AI Mode in regular Search.

Copilot's documentation is the thinnest of the three on mechanics, but Microsoft does publish real citation data through a different channel: Bing Webmaster Tools' AI Performance report, live as a public preview since 10 February 2026, which tracks "how often your content is referenced by AI systems" across "Microsoft Copilot, AI-generated summaries in Bing, and select partner integrations" — though Microsoft itself flags this data as "a sample of overall citation activity," not a complete count.

Which crawlers serve search versus training, for each engine?

EngineCrawler / mechanismWhat it doesRespects robots.txt?
ClaudeClaudeBotCollects content "that could potentially contribute to" model trainingYes
ClaudeClaude-SearchBot"Navigates the web to improve search result quality" — indexing for Claude's search featureYes
ClaudeClaude-UserFetches a page live when a person asks Claude about a specific URLYes
GeminiGooglebot (regular Search crawl) + a separate "grounding with Google Search" retrieval stepStandard Search crawl feeds Google's index; grounding retrieves from Search at answer timeYes (Googlebot); grounding's own crawl behavior isn't separately documented
GeminiGoogle-Extended (a robots.txt token, not a crawler — no user agent ever appears in server logs)Opts a site's already-crawled content out of training and grounding for Gemini Apps and Vertex AI; doesn't affect Search ranking or AI Overviews per Google's own framingN/A — it's read from robots.txt, not a bot that visits
CopilotBingbotStandard Bing crawl; there is no separate, publicly documented "Copilot bot" — Copilot answers draw on Bingbot's indexYes

Unknowns and undocumented are marked explicitly below rather than guessed into this table.

What differs in how each engine actually names and describes brands?

This is the part none of the three document, and it's worth saying so plainly rather than inferring a pattern from a handful of anecdotal chats. What is observable, without claiming to know the mechanism: Gemini's grounding output is structured enough (named source chunks mapped to exact text spans) that, when grounding is on, you can usually tell precisely which sentence came from which URL — a more auditable citation format on paper than a conversational answer with no visible source list. Copilot's citation behavior is now measurable in aggregate through Bing's own dashboard (total citations, which pages, how that changes over time), even though Microsoft explicitly warns that frequency "does not indicate ranking, authority, or the role of any page within an individual answer." Claude's documentation doesn't describe citation-selection logic at all — only crawler purpose and robots.txt behavior.

None of this is enough to rank the three by accuracy or fairness to brands. It only tells you which engine gives you more to inspect after the fact.

What is genuinely unknown, that shouldn't be guessed at?

  • Whether Gemini's search grounding uses the same retrieval index and ranking signals as AI Overviews and AI Mode, or a separate pipeline — Google's grounding documentation doesn't say.
  • The exact reranking or selection logic any of the three use once candidate pages are retrieved — none publish this.
  • How large a role, if any, each engine's own foundation-model training data (as opposed to live retrieval) plays in naming a brand when no citation is shown.
  • Whether Copilot's underlying model weighs sources differently from a standard Bing search ranking once Bingbot's crawl feeds it — Microsoft's documentation describes what's tracked (citations), not how a given citation was chosen.

Treat anything beyond what's quoted above, for any of the three, as inference — including inference in this post.

Which of these should a team actually prioritise?

Here is the honest framing, stated directly rather than implied: ContextOwl's own reports run ChatGPT as the base engine, with Gemini, Perplexity and Claude available as add-ons. Copilot is covered in this post for comparison only, because the pain point — "which platforms to prioritise" — doesn't make sense to answer without mentioning it, but ContextOwl does not report on Copilot or Google AI Overviews as product engines.

With that disclosed, the practical case for the three ContextOwl does cover: Gemini sits inside Google's own ecosystem and increasingly inside AI Overviews and the Gemini app directly, so it matters wherever your buyers already use Google. Claude has grown as a long-form research and work tool, which tends to surface more detailed brand comparisons than a quick chat answer. Perplexity, covered separately in how Perplexity chooses its sources, is worth checking because it shows its source list by default, making it the easiest of the group to audit directly. We lay out a fuller prioritisation framework, independent of which tool you use to check it, in which AI engines should you prioritise.

FAQ

Does Claude have one bot or several? Three: ClaudeBot (training), Claude-SearchBot (search indexing), and Claude-User (user-triggered page fetches). All three respect robots.txt.

Does blocking Google-Extended stop my site from appearing in AI Overviews? No. Google-Extended only controls training and grounding use for Gemini Apps and Vertex AI; it isn't a crawler and, per Google's own framing, doesn't affect Search ranking or inclusion in AI Overviews, which run through Googlebot's regular crawl.

Is there a dedicated Copilot crawler to allow or block? No separate, publicly documented one. Copilot's answers draw on Bingbot's existing crawl and index — the same bot that feeds standard Bing search.

Does ContextOwl track Copilot or Google AI Overviews? No. ContextOwl's reports cover ChatGPT as the base engine, with Gemini, Perplexity and Claude as add-ons. Copilot and Google AI Overviews are not covered.

Which engine is easiest to audit yourself without any tool? Gemini's grounding output and Perplexity's default source list are both more directly inspectable than a Claude or Copilot answer with no visible citations — when either shows a citation, you can usually trace exactly which sentence it supports.

What to do next

  1. Check which of the documented bots — ClaudeBot, Claude-SearchBot, Claude-User, Bingbot — can currently reach your site using the AI Bot Access Checker.
  2. If you use Google's own AI products, separately check Bing Webmaster Tools' AI Performance report for Copilot citation volume — it's free and already live for any verified site.
  3. Decide prioritisation based on where your actual buyers already spend time, not on which engine is easiest to measure — then read which AI engines should you prioritise for the fuller framework.

ContextOwl sells one-time AI visibility reports covering ChatGPT, Gemini, Perplexity and Claude. The AI Citation Snapshot is a free, single-question version of the same check.

Sources

AI Search BasicsClaudeGeminiCopilotcitationsguide

More in AI Search Basics

Be the brand AI recommends.

You've read the research — now see where you actually stand.

Check your AI visibility — free

Your Score is free! Pay per unlock · 90 seconds