Visibility Dropped Overnight: Model Update or Your Fault?

The short version

A drop that hits every prompt on one engine simultaneously is a model or index change. A drop confined to specific prompts across several engines is usually a source that went stale or disappeared.

The shape of the drop identifies the cause before you investigate anything, which is why the first thing to look at is the pattern rather than the number.

Overnight drops feel like emergencies and most of them are not yours. The useful question is not “what did we break” but “what does this pattern rule out,” and two patterns rule out almost everything.

What does the shape of the drop tell you?

Sort the drop along two axes before you do anything else: how many engines it touched, and how many prompts. The four combinations point at four different causes.

Reading the shape of a visibility drop

One engine, every promptA model version or index change on that platform. Not yours.External

Every engine, a few promptsA source those prompts depended on went stale, moved, or dropped out.Yours

Every engine, every promptSomething structural: a crawler block, a site migration, a robots change.Yours

One engine, a few promptsUsually noise. Check the run count before treating it as a drop at all.Probably nothing

Only the second and third rows are worth an engineer’s afternoon. The first is somebody else’s release, and the fourth is often a sampling artefact rather than an event.

The fourth row deserves particular suspicion, because a small drop on one engine is exactly what run-to-run variance produces on its own. Before calling it, confirm the number came from more than one run, which is the first check in how to interpret an AI visibility report.

Did the model change underneath you?

Frequently, and on a published schedule. OpenAI’s deprecations page states that “As we launch safer and more capable models, we regularly retire older models,” with generally available models getting at least six months of notice and preview models sometimes only two weeks. A retirement means the model answering your prompt today is not the one that answered it last quarter.

Even without a retirement, the backend moves. OpenAI’s advanced usage guide exposes a system_fingerprint precisely so callers can detect this, noting that if the value changes “you may see different outputs due to changes we’ve made on our systems.”

If your tracking tool records the model version alongside each answer, a whole-engine drop resolves in about thirty seconds. If it does not, you are guessing.

Did the index change?

For the Google surfaces this is checkable against a public record. Google publishes a Search status dashboard listing every ranking update with its start date and how long the rollout took, and the durations are long enough to matter: the May 2026 core update ran for 11 days and 21 hours.

That matters for AI Overviews and AI Mode specifically, because Google ties them to ordinary Search eligibility. Its documentation on AI features says a page “must be indexed and eligible to be shown in Google Search with a snippet,” and that there are “no additional requirements to appear in AI Overviews or AI Mode.” A core update that moves your Search visibility moves your AI Overviews visibility with it, and the two are not separate problems.

Or did one of your sources go stale?

This is the case that looks like a platform problem and is not. When a drop lands on the same three prompts across four engines, those prompts were leaning on something the engines could all reach, and it changed.

The usual culprits are a review roundup that dropped you in its latest edit, a comparison page that got rewritten, a docs page you moved without a redirect, or a third-party listing that went behind a login. None of them shows up in your own analytics, because the change happened on somebody else’s site.

The tell is that the drop is prompt-shaped rather than engine-shaped. Engines that retrieve differently do not coincidentally lose the same three prompts in the same week unless they were all reading the same page. Separating that from a problem with how engines resolve your brand is the fork covered in entity problem vs content problem AI visibility.

How do you tell them apart in one sitting?

Five checks, in order, and you can stop as soon as one of them explains the shape.

Triage for an overnight drop

Plot the drop by engine and by promptTwo columns, one row per prompt. The shape is visible before any analysis and decides everything after it.

Confirm it is not samplingRe-run the affected prompts five times today. If today’s five disagree with each other, yesterday’s single run was never a measurement.

Check the model versionCompare the recorded model or fingerprint before and after. A change here explains a whole-engine drop and ends the investigation.

Check the public update recordFor the Google surfaces, match your date against the Search status dashboard. An overlapping core update is a sufficient explanation.

Read the answers that replaced youFor a prompt-shaped drop, look at what the engines now cite. The source that displaced you names the page that changed.

Step five is the only one that requires reading rather than counting. It is also the one that produces an action, because the replacement source tells you what to fix or who to talk to.

Step five is worth dwelling on. When a competitor appears where you used to, the answer text usually reveals which page carried them in, and that page is either one you can be added to or one you can beat. The reading method is the same one in AI recommends competitor instead of us.

What if the shape is ambiguous?

Widen the prompt set rather than staring harder at the same one. A drop that looks prompt-shaped across six prompts often resolves into engine-shaped across thirty, because six was never enough to see the pattern. The unbranded set in prompts to test AI brand visibility is a reasonable baseline to widen into.

And write down the date with the model version next to it. The single most useful artefact in this whole exercise is a log that lets next quarter’s you rule out this quarter’s cause in one glance.