More and more buyers now ask an AI assistant instead of scrolling a results page. The assistant answers, names a few sources, and those sources get the visit. Whether your site is one of them is decided by things you control.

The shift is not hypothetical. When someone asks an assistant "who does website monitoring in Denmark" or "what does a TLS certificate cost", the assistant does not recite a memorised list. It runs a search, fetches a handful of pages live, reads them, and writes an answer that names its sources. Those named sources get the click, the credibility and — often — the enquiry.

That makes visibility in AI answers a distribution channel, not a curiosity. It also makes it unusually winnable, because most sites have never checked the four settings that decide it. This guide covers what those settings are, how to tell where you stand, and why the most common mistake is one nobody notices from inside a browser.

What is generative engine optimisation (GEO)?

Generative engine optimisation is the practice of making a website usable as a source by AI answer engines such as ChatGPT, Perplexity, Google AI Overviews and Gemini. It overlaps heavily with ordinary SEO — the page still has to be reachable, readable and relevant — but it adds a distinct requirement: the answer to a question must be extractable in one self-contained passage, because that is the unit an assistant lifts and cites.

That last point is the practical difference. A traditional search result rewards a page that covers a topic well overall; the visitor arrives and does the reading. An assistant does the reading itself, then quotes. If your explanation of a thing is spread across four paragraphs, an introduction and a table, there is nothing clean to lift. If it sits in three sentences directly under a heading that matches the question, there is.

So the writing advice is unglamorous and specific: put the answer first, keep it short enough to quote, and make it stand on its own without the sentences around it. Everything else on the page can be as long as it needs to be.

Do AI assistants use the same crawlers as Google?

No. Assistants send their own user agents, and each operator runs more than one. OpenAI alone uses GPTBot for training data, OAI-SearchBot for its search index and ChatGPT-User when a person asks a question that needs a live fetch. Allowing Googlebot says nothing about whether these are allowed, because robots.txt rules are matched per user agent.

This is where a lot of sites lose without ever deciding to. A robots.txt written years ago, a security plugin with a "block bad bots" setting, a CDN rule added during a scraping incident — any of them can exclude an assistant while leaving Google untouched. Nothing breaks, nothing is logged anywhere the marketing team looks, and the site simply stops being quotable.

The list of tokens is also unstable. Operators add, rename and retire them with no notice and no central registry, so a check that was accurate a year ago is not accurate now. This is why TrustCtrl treats a crawler it does not recognise as unknown rather than allowed: claiming coverage the check does not have would be worse than admitting the gap.

Does blocking AI training stop me appearing in AI answers?

Not necessarily, and the distinction is the most useful thing to understand about GEO. Training crawlers such as GPTBot, ClaudeBot and CCBot collect text that may feed a model. Retrieval crawlers such as OAI-SearchBot, PerplexityBot and Claude-User fetch a page to answer a question being asked right now, and they are the ones that produce a citation. Blocking training while allowing retrieval is a deliberate and entirely valid position: your content stays citable without feeding anyone's model.

Plenty of businesses want exactly that. They are happy to be a cited source — it brings traffic and authority — and unhappy about their writing becoming training data for a model that will answer without them next time. That is a coherent position, and it is expressible in robots.txt.

The trouble is that the two categories are easy to conflate. A single Disallow aimed at "AI bots" usually catches both, which quietly converts "don't train on us" into "don't cite us either". TrustCtrl reports the two groups separately for this reason, so the page tells you which choice you have actually made rather than lumping them into one number.

  • Training crawlers: GPTBot (OpenAI), ClaudeBot (Anthropic), CCBot (Common Crawl), Bytespider (ByteDance), Meta-ExternalAgent (Meta).
  • Retrieval crawlers: OAI-SearchBot and ChatGPT-User (OpenAI), PerplexityBot (Perplexity), Claude-User (Anthropic), Applebot (Apple).
  • Robots-only switches: Google-Extended and Applebot-Extended are opt-out controls for Gemini and Apple Intelligence training. No crawler ever sends them as a user agent — they exist purely as a robots.txt signal honoured by crawlers that identify as something else.

That third category catches people out. Because nothing ever arrives calling itself Google-Extended, testing for it by watching your logs proves nothing. It has to be read out of robots.txt.

How do I get my website cited by ChatGPT?

There is no submission form and no guarantee. In practice three things decide it: the retrieval crawlers must be allowed to fetch your pages, the answer must be present in the HTML the crawler receives rather than assembled later by JavaScript, and the page must answer a real question in a short, self-contained passage. Sites that fail any one of them are invisible no matter how good the content is; meeting all three is what puts you in the running.

The middle condition is the one that catches modern sites. Many assistants read the HTML as it arrives from the server. If your product description, your prices or your FAQ answers are injected by a framework after the page loads, the crawler may receive a near-empty shell — a navigation bar, a footer and a loading state. The page looks perfect in your browser, because your browser ran the JavaScript. The crawler did not.

This failure is invisible from the inside, which is why it survives for years. TrustCtrl compares what the server returns with what the page looks like after rendering and flags pages where the main content only exists in the second version. We cover the mechanics in the content Google cannot see.

The fourth setting: pages that answer differently to a crawler

There is one more failure mode, and it is the one most likely to be doing damage without anyone knowing. Some sites return different content depending on who is asking — a bot-protection layer serving a challenge page, a paywall that opens for Googlebot but not for anyone else, a CDN rule that treats unfamiliar user agents as suspicious.

Search engines call this cloaking, and when it is deliberate it is a ranking risk. When it is accidental, which is far more common, the effect is simpler: the assistant receives a security-check page instead of your article, and cites someone else. TrustCtrl probes your pages as each AI crawler in turn and reports when the response differs materially from what an ordinary visitor gets, which is the only way to catch a rule you did not know was there.

What about llms.txt?

You will see llms.txt recommended as the GEO checklist item. It is a proposed file at your site root that tells an assistant, in one page of markdown, what your site is and which URLs are worth reading. It is a sensible idea and it costs almost nothing to publish.

It is also, today, immature: no search engine or assistant is known to require it, and adoption is thin. Treat it as cheap insurance rather than a fix. We go through what to put in one — and the far more consequential thing a missing file can reveal about your server — in the llms.txt guide.

How do I check whether AI assistants can read my site?

Fetch your own robots.txt and read it per user agent rather than assuming a blanket rule applies, then confirm that the answer text is in the raw HTML and not only in the rendered page. TrustCtrl does both on every crawl: it reports each AI crawler separately as allowed, blocked or unknown, splits them by training and retrieval, and flags pages whose main content only appears after JavaScript runs.

Doing it by hand once is worthwhile and takes an afternoon. The reason to automate it is that none of these settings stay put. Robots.txt gets edited during an unrelated incident, a CDN rule is added by whoever is on call, a framework upgrade moves content from server-rendered to client-rendered, and a new crawler token appears that your old rules do not mention. Each change is small, none of them announces itself, and the first symptom is traffic that quietly stops arriving.

TrustCtrl visits your site in a real browser on a schedule, checks the AI-visibility settings alongside the rest of your site quality — titles and descriptions, sitemap accuracy, speed, accessibility and security settings — and tells you in plain language what changed and what to do about it. One crawl, one picture — written plainly enough for the owner to act on, with the technical detail underneath for whoever makes the change.