Blog
6 min read AEO Fundamentals

How ChatGPT Chooses Sources: Browsing, Memory, and Training Data

On this page
  1. What are the three stores?
  2. What decides whether it browses?
  3. Which crawler feeds the browsing store?
  4. Is ChatGPT unusual in working this way?
  5. How do you tell which store answered you?
  6. What does this mean for what you track?
  7. Can you make it browse more often?
The short version

ChatGPT answers from three separate stores, and which one it uses depends on the phrasing of the question rather than a setting. Product questions usually trigger browsing; category questions often do not.

You cannot control which store it reaches for, and you can tell which one it used from the answer itself. That is enough to know whether this quarter’s work can reach a given question at all.

People ask what ChatGPT “knows” about their company as though there were one answer. There are three, they update on completely different timescales, and the same product gives you a different one depending on how the question is worded.

What are the three stores?

Three places an answer can come from
1

Training data

What the model absorbed before its cutoff.

Updates on
Model releases. Not your schedule.
Tell it by
No sources cited, confident tone, possibly out of date.
2

Browsing and search

Pages fetched during the request.

Updates on
Crawl cycles. Days, sometimes faster.
Tell it by
Inline citations and links you can click.
3

Memory

What this user told it in earlier conversations.

Updates on
That one person’s usage.
Tell it by
Suspicious familiarity with your company.
Only the middle one is reachable by anything you do this quarter. The third is the reason your own testing lies to you if you do not disable it.

The third store deserves a warning rather than a strategy. Memory is per-user, so it inflates your own results and nobody else’s. If you have discussed your company in that account before, the engine may name you for reasons no buyer will ever benefit from. Test in a temporary session with memory off, always.

What decides whether it browses?

The question, not a toggle. And the mechanism is documented, if not by OpenAI in detail then by a directly comparable system: Anthropic states plainly that “Claude determines when to search based on the prompt”, and searches when a request “depends on information that is current, changing, or outside its training data” while answering directly from stable knowledge otherwise.

That is the shape of the behaviour across this class of product. Something in the question signals currency, specificity or uncertainty, and the system looks things up. Something stable and general, and it does not bother.

Often answered from memory “What is field service software?”

Stable, general, unchanged for years. Nothing about your last quarter can reach an answer like this.

Usually triggers a lookup “Best field service software for a 12-person team, with current pricing”

Specific, constrained, and asking for something that changes. This is the question shape your work can influence.

Prompt shape decides which store you are testing. A prompt set full of general questions will report that nothing you did mattered, and it will be measuring the wrong store.

This is worth stating as a caution rather than a rule, because the trigger conditions are not published in detail and they change. Treat it as something you verify per prompt rather than a law you can rely on.

Which crawler feeds the browsing store?

A specific one, and it is separate from the one people usually think about.

OpenAI documents that OAI-SearchBot is “used to surface websites in search results in ChatGPT’s search features”, while GPTBot collects data for model training and ChatGPT-User fetches pages when a person asks for one directly. Crucially, the settings are independent: a webmaster “can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot”.

OAI-SearchBot Feeds store 2. This is the one that decides whether browsing can find you at all. Allow
GPTBot Feeds store 1, on a model-release timescale. Blocking it is defensible and does not affect search results. Your call
ChatGPT-User Fetches a page because somebody named it. Robots rules may not apply to this class of visit. Ignores robots.txt
One file governs all three and they are configured separately. A blanket rule written to opt out of training is the most common way sites remove themselves from store 2 by accident.

Worth knowing before you edit anything: OpenAI notes that a robots.txt change can take around 24 hours to register on their side, so a fix made this evening is not a result you can test tonight.

Is ChatGPT unusual in working this way?

No, and that is useful, because it means what you learn here transfers.

The three-store structure is close to universal among assistants that can search. What differs is the mix and the documentation. Perplexity is built search-first and documents the same crawler split, between a bot that surfaces and links sites and a fetcher for user-initiated visits. Google’s AI surfaces are grounded in the Search index rather than a purpose-built crawl, and Google states there are “no additional requirements to appear in AI Overviews or AI Mode” beyond ordinary Search eligibility.

Where ChatGPT is genuinely distinctive is the memory store, which is per-user and therefore invisible in anybody else’s results. No other major engine gives an individual tester such an easy way to fool themselves.

Transfers

The store structure, the crawler split, and the fact that prompt shape decides which store answers.

Does not transfer

The exact trigger conditions, which are undocumented and change, and the memory behaviour, which is ChatGPT’s own.

Learn the structure here, then verify the specifics per engine. The architecture is stable enough to reason about; the routing rules are not.

How do you tell which store answered you?

Read the answer rather than the content of the answer. Three tells, none of which require any tooling.

Citations present

It browsed. You are in retrieval territory, the sources are inspectable, and your work can reach this question.

No citations, confident tone

Probably training data. Check whether the facts are stale, because that is the signature failure of this store.

Unearned familiarity

It knows things about your setup that a stranger would not. That is memory, and your test is contaminated.

Do this before interpreting any result. A finding from store 1 and a finding from store 2 call for completely different responses, and they look identical if you only read the text.

What does this mean for what you track?

Two things, and both change how you build a prompt set.

Write prompts that trigger the store you can actually influence. Specific, current, constrained buyer questions reach browsing. General definitional questions reach training data, and measuring those is measuring something you cannot move for another model generation.

And keep the full answer text rather than a tally, because the tells above live in the text. A yes-or-no record of whether you were mentioned cannot tell you afterwards whether the mention came from a page you can fix or from a store you cannot touch.

The three-prompt version of this check is in the five-minute test, the wider retrieval architecture is in the RAG explainer, and how the engines split into families that behave this way is in the engine breakdown.

Can you make it browse more often?

Not directly, and the honest framing is that you influence the question rather than the engine.

Nothing you publish changes whether a given prompt triggers a lookup. What you can do is recognise that your buyers ask specific, constrained, current questions far more often than they ask definitional ones, so the questions that matter commercially are already the ones most likely to browse. The general definitional questions that answer from stored knowledge are also the ones least likely to end in a purchase.

That is a more comfortable position than it first sounds. The store you can reach is the store your buyers are using at the moment they are deciding, and the store you cannot reach mostly serves people who are still learning what the category is.

One caveat about this whole page. OpenAI documents its crawlers thoroughly and its answer-routing behaviour barely at all, so the browsing and memory mechanics above are read from documentation where it exists and from observable behaviour where it does not. Treat the crawler facts as settled and the routing behaviour as current best reading.