How sampling works
What one cycle does, what it costs, and how chats become readings.
Every number in Recomma is downstream of one loop. It is worth understanding because it explains both what the figures mean and what the project costs.
One cycle
On each cycle, for every tracked prompt, on every enabled model:
Ask
The question is put to the model — through its own interface where possible, through the model’s API as a fallback. See fidelity.
Store the chat whole
The text, the brands it named in the order it named them, and every source it cited or read. The chat is kept, not just a score derived from it, which is why every figure can be opened and read back.
Count
Visibility, share of voice, sentiment and position are counted from the stored chats and rolled up per day, per model, per market.
Cadence
A project samples daily or weekly.
- Daily while you are actively working on visibility — you want to see a change land.
- Weekly when you are watching a trend. Four times fewer chats, and a reading that is four times slower to become trustworthy.
What drives the volume
chats per cycle = tracked prompts × enabled modelsThat is the whole formula, and both terms are yours to choose. Forty prompts on six models is 240 chats a cycle; daily, that is about 1,700 a week.
This is why suggested prompts wait for a person before they start running. Coverage should grow because somebody decided it should.
Fewer prompts, more often, beats more prompts rarely
Sample size is what makes a percentage a reading, and it accumulates per prompt over time. Twenty questions sampled daily reach a trustworthy reading far sooner than eighty sampled weekly — and you will actually read twenty.
From chats to a reading
A single chat is a coin flip. The metrics only mean something in aggregate, which is why Recomma is explicit about the aggregate everywhere:
- Sample size — prompts × models × sampling days in the window.
- Confidence interval — the range the true figure is likely to sit in.
- Below twenty chats, the figure is greyed out. Not wrong — just moving too much on one chat for a difference between two readings to mean anything.
What breaks comparison
A trend line is only comparable over a period where the measurement did not change. Three things change it:
| Change | Effect |
|---|---|
| Adding or removing prompts | Moves the visibility baseline — different questions |
| Adding or removing competitors | Moves share of voice and position — different field |
| Enabling or disabling an model | Moves everything — different mix of chats |
None of these is wrong to do. But expect a step in the line at the point you did it, and do not read that step as the market moving.