How we'll know if Jonbar is right.

A real, past commercial decision with a public outcome: a price change, a campaign mechanic, a launch price. It needs at least two options (what was done and the alternative) and a reported result on a named metric (units, revenue, margin).

1. 01

   Split and seal.

   The case is split in two: what was known before the decision, and the outcome. The outcome is sealed; only its fingerprint (a hash) is stored.

2. 02

   Run on the "before" part.

   The engine runs on the "before" part only, at a fixed version.

3. 03

   Lock the prediction.

   The prediction is committed with its version and seeds before anyone opens the outcome.

4. 04

   Open and score.

   We unseal the outcome and score direction and size of the miss. Any change to the engine after that is a new iteration, tested on new sealed cases.

5. 05

   "I can't tell" counts as wrong.

   You can't score well by skipping hard cases.

• +Direction: did we pick the option that actually did better?
• +Size of the miss: how far our predicted effect was from the real one, in percentage points.

Neither is shown alone. Direction on its own is a weak bar: even simple rules often get it right.

• +The textbook rule: price up → units down; discount up → sales up.
• +A fixed average elasticity.
• +A plain AI language model asked "which option won?", with no simulation (planned; measured once enabled).

We report Jonbar's score next to theirs on the same cases. We only say "better" if the gap is statistically significant.

The first sealed test set will be small, so the first number will be a rough signal, not proof. We say so on the result, with its range.

In a pilot we pick one of your past campaigns, see only data from before it started, lock the prediction, and then compare it with what really happened. This is also how accuracy on Turkish marketplace data gets measured: it hasn't been yet.

Each run shows a range for its result. That range is the spread under the model's own assumptions, not a probability that reality lands inside it. We'll tell you how often real results fell inside these ranges once the backtest is published.

The result, good or bad: direction score, median miss, how often real results fell inside the ranges, the simple-rule scores, and every high-confidence miss listed by name of case.