A usable prompt set is fixed, unbranded, and derived from real buyer language rather than keyword tools. Changing the set mid-quarter destroys comparability, which is the only thing tracking gives you.
The set is not a list of questions. It is a versioned artefact with an owner, and treating it as anything less is why most tracking programmes produce numbers nobody trusts by month three.
Everything downstream depends on this one object. A prompt set that drifts produces a chart where every movement is ambiguous, and no amount of sampling rigour rescues it.
What is a prompt set, as an artefact?
A file, under version control, containing the exact questions you track and the rules for reading the answers. Four fields per prompt, and none of them is optional.
| Field | Why it is in the file |
|---|---|
| The prompt, verbatim | Punctuation included. A reworded prompt is a different prompt and starts a new series. |
| Branded or unbranded | The two get reported separately and never averaged, so the label has to live with the prompt. |
| Intent | Category, comparison, constraint, problem, or brand. Lets you report by segment rather than as one blended figure. |
| Date added, and by whom | A prompt added in week six explains a step change in week six. Without the date, it looks like a market movement. |
Storing this in a spreadsheet is fine. Storing it only inside a tool’s settings is not, because a settings page has no history and anyone with a login can quietly change the basis of your reporting.
Where do real prompts come from?
Records of things people said, not models of what they might search. Google’s May 2026 post on AI Mode reports that “the average AI Mode search is triple the length of a traditional Search query,” with growth in phrasings like “where to” and “where should I.”
That length is the tell. Keyword tools return the compressed forms people typed at a search box, and a compressed form has had exactly the constraints stripped out of it that make a question answerable by a specific product.
Microsoft’s AI Performance report in Bing Webmaster Tools is the only free source of the first kind. It shows “the key phrases the AI used when retrieving content that was referenced in AI-generated answers,” which is a record of retrieval rather than an estimate of demand.
How many prompts, and split how?
Enough to cover the intents, few enough that every one earns its place. For a single category, 25 to 50 is the range where most teams land, and the split matters more than the count.
Test the set with a constraint count. Add up the constraints across your prompts, divide by the number of prompts, and if the average is below one you have a keyword list wearing sentences.
Why does changing the set destroy the value?
Because tracking gives you exactly one thing that a single audit does not, which is comparability over time. Edit the set and you have two audits with different bases and no way to difference them.
A prompt set that changes every quarter produces a chart with a shape and no meaning. The shape is your editing history.
This compounds with a subtler problem. Adding an easy prompt raises your average, and adding a hard one lowers it, so an owner under pressure can move the number without touching the market. That is not usually deliberate, and it is indistinguishable from a real trend on the chart.
So how do you ever change it?
With a version number and a dual-run period, the same way you would change any measurement instrument.
Freeze version one and keep running it. Start version two alongside. Run both for one full reporting period, publish both numbers with the change explained, then retire version one. The overlap is what makes the two series comparable, and skipping it is why most changes look like results.
Add prompts only at a version boundary. Nothing is more corrosive to a programme than prompts appearing mid-quarter because somebody thought of a good one, and the fix is a queue rather than a prohibition.
What has to be true before the set is worth running?
Three things, and only the third costs money.
Every prompt is unbranded except the labelled control group. Branded prompts test whether an engine knows you once you have named yourself, which is a question you can usually answer without a programme.
Every prompt has at least one constraint. Without one you are measuring the market leaders’ position, which is stable, well understood, and not about you.
And the run count is decided in advance. OpenAI’s advanced usage guide notes that “Chat Completions are non-deterministic by default (which means model outputs may differ from request to request),” so a single run per prompt gives you 0% or 100% rather than a rate. Deciding the count after seeing the results is how a set becomes an argument.
Who owns the file?
One named person, and the answer matters more than it sounds. A prompt set with no owner accumulates edits from everyone with access, and each edit is individually reasonable.
The owner’s job is narrow: refuse mid-quarter changes, run the version boundary, and keep the change log. They do not need to write the prompts, and arguably should not, since the best ones come from sales and support rather than from marketing.
The failure mode without an owner is specific and common. Somebody notices the set is missing an obvious question, adds it, and the next report shows a movement nobody can explain because the addition is not written down anywhere. Six weeks later the programme has a number that senior people have learned not to trust, and no way to rebuild the trust without starting the series again.
Does the same set work on every engine?
Yes, and it must, or the comparison between engines breaks. Run the identical set everywhere and report per engine.
Expect the results to differ substantially, because the engines do. Google states that even its own two surfaces diverge, noting in its documentation on AI features that “AI Mode and AI Overviews may use different models and techniques, so the set of responses and links they show will vary.”
That divergence is a finding rather than a problem. A prompt where you appear on three engines and not a fourth points at something specific about the fourth, and you can only see that if the prompt was identical everywhere. Which engines to run it against is a separate decision, and the sizing question behind it is covered in enterprise ai visibility platform. How the two prompt types get reported is in branded vs unbranded prompts ai visibility.