Blog
6 min read AEO Fundamentals

What Is Retrieval Augmented Generation and Why It Decides Your Visibility

On this page
  1. What problem was RAG built to solve?
  2. What actually happens when someone asks a question?
  3. What does the retriever actually match on?
  4. What happens at generation?
  5. What is a chunk, and why should you care?
  6. Why do two engines answer differently?
  7. What does this change about what you do?
  8. What does RAG not solve?
The short version

RAG means the model fetches live documents before answering. Your visibility therefore depends on two separate things: whether the retriever finds you, and whether the generator chooses to name you.

Almost every practical question in AI visibility resolves to which of those two is failing, and the fixes have nothing in common.

Retrieval-augmented generation is the least glamorous acronym in this field and the most useful thing to understand about it. It is the difference between a system that can only tell you what it memorised and one that goes and looks. That second behaviour is the entire reason AEO is possible: if models only ever recited training data, nothing you did this quarter could change what they say about you.

What problem was RAG built to solve?

Models store knowledge in their weights, and that storage is both frozen and unreliable to query. The paper that named the pattern, Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, opens on exactly that observation: a pre-trained model’s “ability to access and precisely manipulate knowledge is still limited”.

The fix was to stop relying on memory alone. The system keeps the model’s stored knowledge and adds a lookup over an external corpus at query time, so the answer can be assembled from documents rather than recalled.

Parametric memory What it learned in training

Fixed at training time and not updatable without retraining. If your company changed after the cutoff, this half of the system does not know.

Non-parametric memory What it looks up right now

An external corpus fetched during the request. Live, changeable, and made of documents that exist on the open web today.

AEO operates almost entirely on the right-hand side. You are not changing what a model learned. You are changing what it finds when it looks.

What actually happens when someone asks a question?

The sequence, and where you can intervene
Question

Asked in natural language, often with constraints the buyer cares about.

Retrieval

Passages scored against the question, top few returned.

Be findableBe self-containedBe crawlable
Generation

An answer written from what came back, with sources attached where grounding worked.

Be describableBe agreed upon
Two intervention points, and they need different work. The left one is technical and lives on your domain. The right one is positioning and lives everywhere else.

The critical property is ordering. Retrieval runs first and it is a hard filter: the generator can only write from what it received. Excellent material that was not retrieved is indistinguishable, from the answer’s point of view, from material that does not exist.

What does the retriever actually match on?

Meaning, increasingly, rather than words. Karpukhin et al. demonstrated dense retrieval, where questions and passages are embedded into vectors and matched by proximity, outperforming a strong term-matching baseline by 9 to 19 points on top-20 passage accuracy.

Two practical consequences follow, and they point in opposite directions from a decade of habit.

Repeating a phrase buys less

If a passage can be matched without containing the query’s words, saying them more often adds little that meaning did not already carry.

Passage structure buys more

Retrieval scores chunks. A section that stands alone is a candidate; one that needs its neighbours to make sense is usually not.

Write for the passage, not the page. It is the single highest-return on-page change available, and it costs nothing but discipline.

What happens at generation?

The model writes an answer and attaches sources where it can tie a statement to a retrieved document. Anthropic documents this directly: for its web search tool, “citations are always enabled”, and each one carries the URL, the title, and up to 150 characters of the text actually used.

That last detail is worth sitting with. The citation is anchored to a passage, not to a page. The engine is not endorsing your article. It is saying that this specific sentence came from here, which is why a page that answers one thing precisely gets cited more often than one that covers ten things well.

What is a chunk, and why should you care?

A chunk is the passage-sized unit a page gets split into before it is indexed. Engines retrieve chunks, not pages, and that one fact carries more practical weight than anything else in this article.

It means the thing competing for inclusion in an answer is not your carefully structured 2,000-word guide. It is one section of it, judged alone, by a system that never saw the introduction.

A chunk that fails “As described above, the second approach avoids this.”

Retrieved out of context it says nothing. It cannot be scored well, and if it somehow is, it cannot be quoted.

A chunk that works “Dense retrieval matches on meaning rather than shared words, so a passage can be found without containing the query’s terms.”

Complete on its own. Scoreable, quotable, and attributable to you.

Every H2 section should survive being read in isolation. That single editing rule is most of what “optimising for AI” honestly amounts to.

The habit this replaces is writing a page as one continuous argument. Continuity is still good for the human reading top to bottom; it just needs to be built out of parts that each stand up alone, rather than out of sentences that depend on the ones before them.

Why do two engines answer differently?

Because each runs its own retrieval over its own corpus. There is no shared index and no shared rulebook.

OpenAI runs OAI-SearchBot for the index behind ChatGPT’s search features, kept separate from its training crawler, and Perplexity documents the same split for PerplexityBot. Google’s AI surfaces draw on the Google index with ordinary Search eligibility. Those are three different corpora assembled by three different crawlers, so disagreement between them is the expected result rather than a bug in your measurement.

If two engines disagree about you, they are not both wrong. They looked in different places and found different things, which is itself a finding about where your evidence lives.

What does this change about what you do?

It gives you a way to tell two failures apart, which is the whole practical payoff.

Retrieval failure Nothing about you comes back, or the engine describes you vaguely and hedges. Crawler access, missing pages, or content that does not survive being chunked. Cheap to fix
Generation failure It describes you well and still does not name you when asked to recommend. The sources it retrieved do not agree about you, or barely mention you at all. Slow to fix
These are the only two options, and one afternoon separates them. Buying a fix before you know which one you have is how a quarter disappears.

The mechanics of the second stage, and why consistency beats prominence there, are in how models decide which brands to recommend. If you want to find out which failure you have before reading anything else, the five-minute test settles it.

What does RAG not solve?

Two things, and both matter for how much you should trust an answer about your own company.

It does not guarantee the answer is grounded. Retrieval improves the odds that a claim traces to a document, and the generator can still produce a statement no retrieved source supports. That is why a stale price or a removed feature can appear confidently in an otherwise well-sourced answer, and it is why “the engine cited three sources” is not the same as “everything it said came from them”.

It also does not fix a corpus that is wrong about you. If the documents the retriever finds all repeat an outdated description, retrieval works perfectly and the answer is still wrong. The failure is upstream of the architecture, and the fix is on somebody else’s page rather than in anything a model does.

Both are reasons to read the full answer text rather than a mention count. A tally records that you were named; only the text records that you were named as something you stopped being two years ago.

The acronym is worth remembering for one reason only. RAG is why this is a discipline rather than a lottery: the documents are fetched fresh every time, which means what they say about you is still changeable.