Without a baseline taken across at least two weeks and multiple runs per prompt, you cannot distinguish the effect of your work from normal variance. Answers differ between identical requests, so a single before-and-after pair is not evidence of anything.
A baseline is not a number. It is a recorded protocol plus the spread that protocol produced while nothing was changing.
The temptation is to measure once, ship the fix, measure again, and report the difference. That comparison is unfalsifiable, because the same two measurements would have differed even if you had done nothing in between.
Why can you not just measure before and after?
Because the measurement moves on its own. OpenAI’s advanced usage guide states that “Chat Completions are non-deterministic by default (which means model outputs may differ from request to request),” and even a fixed seed produces only mostly deterministic output.
Google says the same about its surfaces from a different angle. Its documentation on AI features notes that “AI Mode and AI Overviews may use different models and techniques, so the set of responses and links they show will vary.”
So the question a baseline answers is not “where are we.” It is “how much does this number move when nothing is happening,” and you cannot know whether a change is real until you know that.
How long should a baseline run?
Two weeks at minimum, and the reason is not statistical. It is that platform-side changes are common enough that a shorter window can sit entirely inside one.
Google publishes a Search status dashboard recording every ranking update with its start date and rollout duration, and those durations are substantial: the May 2026 core update ran for 11 days and 21 hours. A one-week baseline taken across that window is measuring the rollout, not your brand.
Two weeks also gives you day-of-week structure. If your category has weekday and weekend behaviour, a five-day baseline will mislead you about the range.
How many runs per prompt, and on how many engines?
More than one run, and every engine you intend to report on. A single run per prompt gives you 0% or 100% for that prompt, which is not a rate and cannot anchor a comparison.
Engines belong in separate rows from the start. Baselining a blended figure and then trying to decompose it later does not work, because you have no per-engine history to decompose into. The full argument for keeping them apart is in how to calculate AI share of voice.
What exactly do you record?
Six fields per observation. The first three are the measurement and the last three are what makes it reproducible later.
Recording the answer text rather than only the score is what lets you re-code the data later. If you decide in month three that being listed as a runner-up should not count as a mention, you can recompute the whole baseline. If you stored scores, you cannot.
How do you know the baseline is stable enough to use?
Compute the spread, not the average. Take your per-engine figures across the two weeks and look at the distance between the highest and lowest observation while nothing was changing. That distance is your noise floor.
Any future movement smaller than that floor is not a result. It might be real, but your instrument cannot see it, and reporting it as an effect is how programmes lose credibility with a finance team that can read a chart.
The output of a baseline is a threshold. Below it you say nothing; above it you investigate.
If the spread is uncomfortably wide, widen the prompt set rather than extending the window. Breadth reduces uncertainty faster than depth on any one prompt, because prompts vary more than runs of the same prompt do.
When can you start changing things?
Once the protocol is frozen and written down. Freezing means the prompt set, the competitor list, the run count and the engine set do not change again while you are attributing anything to your work.
Then change one thing at a time. Shipping a schema deployment, a new comparison page and a docs restructure in the same week produces one movement and three candidate causes, and you will never know which.
That last distinction matters more than it sounds. Work aimed at retrieval and work aimed at the model’s memory of you resolve on completely different clocks, which is the split covered in mention rate vs citation rate.
Who needs to agree to the freeze?
Whoever can ship something that moves the number, which is usually more people than the person running the measurement. A content team publishing on its own schedule, a web team deploying a template change, and anyone updating third-party listings can all break attribution without knowing a baseline exists.
The practical version is a shared change log rather than a moratorium. Nobody has to stop working; they have to write down what went out and when. A dated line in a file is enough, and it converts an unanswerable argument into a lookup.
Two weeks of genuine freeze is better and rarely achievable. Take the change log if that is what you can get, and record which of the two you actually had, because it changes how confidently the results can be read later.
What if you already started fixing things?
Baseline now anyway, and mark the start date honestly. A baseline taken after the work began is still a valid noise floor for everything you do next, and it is worth more than an argument about what the number was in June.
Do not reconstruct a historical baseline from a tool that only stored scores. You cannot recompute what you cannot re-read, and a reconstructed figure carries a confidence it has not earned. The reading discipline for numbers you did not generate yourself is in how to interpret an AI visibility report, and if the current figure looks wrong before you start, why is my brand not showing up in AI search is the place to begin.