How much this moved since the previous period of the same length, in percentage points. “Within noise” means the move is smaller than the ± range, so it could be chance.AI engines do not give the same answer twice. Some movement from one period to the next is just that, and Citino will not dress it up as a trend.
How a change is shown
- In points. A change is the difference between two percentages, in percentage points (
pts), never the percentage increase on the first figure. - Against the previous period of the same length. On the 30-day window, the last 30 days are compared with the 30 days before them.
- New questions are left out of the comparison. A question that was not yet running in the earlier period is left out of both sides, so adding questions does not look like a jump.
If a number has no
± range to compare with, any change in it is treated as within noise. Not knowing how wide the uncertainty is counts as “not news”.
What “within noise” stays out of
A change within noise is never treated as a move anywhere in the product:- The digest. Overview’s what-moved list shows “Only the changes big enough to be real rather than chance. If nothing is listed, nothing moved by more than its ± range.”
- Alerts. “A change small enough to be chance never raises an alert.”
- Sorting. It never sorts as a movement.
What to do with a change within noise
- Do not report it as a win or a loss. It may well be real, but the data cannot tell it from chance yet.
- Look at a longer window. More answers usually narrow the range.
- Check the
±itself. The range is worked out across your questions, so when there are few of them, adding questions narrows it as well as waiting does. See The ± band and n.
Related
The ± band and n
Where the range comes from.
Drift labels
When the whole engine moves at once.
Alerts
What does raise an alert.
How bands are computed
The method behind the range.