How sampling works
What one sampling cycle does, what it costs in chats, and how stored chats become readings with a sample size and a confidence interval.
Every number in Recomma is downstream of one loop. It is worth understanding because it explains both what the figures mean and what the project costs.
One cycle
On each cycle, for every tracked prompt, on every enabled model:
Ask
The question is put to the model — through its own interface where possible, through the model’s API as a fallback. See fidelity.
Store the chat whole
The text, the brands it named in the order it named them, and every source it cited or read. The chat is kept, not just a score derived from it, which is why every figure can be opened and read back.
Count
Visibility, share of voice, sentiment and position are counted from the stored chats and rolled up per day, per model, per market.
Cadence
A project samples daily or weekly. Every plan runs daily; weekly is there for a project you would rather spend fewer chats on.
- Daily while you are actively working on visibility — you want to see a change land.
- Weekly when you are watching a trend. Four times fewer chats, and a reading that is four times slower to become trustworthy.
What drives the volume
chats per cycle = tracked prompts × enabled modelsThat is the whole formula, and both terms are yours to choose within the plan’s limits — Starter and Growth enable at most three of the surfaces, and the plans above them are uncapped.
Forty prompts on three surfaces is 120 chats a cycle; daily, that is about 840 a week. The same forty on all five is 200 a cycle and roughly 1,400 a week.
This is why suggested prompts wait for a person before they start running. Coverage should grow because somebody decided it should.
Fewer prompts, more often, beats more prompts rarely
Sample size is what makes a percentage a reading, and it accumulates per prompt over time. Twenty questions sampled daily reach a trustworthy reading far sooner than eighty sampled weekly — and you will actually read twenty.
From chats to a reading
A single chat is a coin flip. The metrics only mean something in aggregate, which is why Recomma is explicit about the aggregate everywhere:
- Sample size — prompts × models × sampling days in the window.
- Confidence interval — the range the true figure is likely to sit in.
- Below twenty chats, the figure is greyed out. Not wrong — just moving too much on one chat for a difference between two readings to mean anything.
What breaks comparison
A trend line is only comparable over a period where the measurement did not change. Three things change it:
| Change | Effect |
|---|---|
| Adding or removing prompts | Moves the visibility baseline — different questions |
| Adding or removing competitors | Moves share of voice and position — different field |
| Enabling or disabling an model | Moves everything — different mix of chats |
None of these is wrong to do. But expect a step in the line at the point you did it, and do not read that step as the market moving.