Improve
Would a different rule have caught it?
Re-run any past request under another policy version and get a signed answer, next to what people actually decided.
Under refunds v12
What happened- Rule
- Over £500
- Route
- Two reviewers
- People decided
- Approved
Under refunds v13
What would happen- Rule
- Over £750
- Route
- One reviewer
- Signed answer
- Valid
Why it matters
What counterfactual replay changes
- Re-run any past request under another policy version.
- Get a signed answer.
- See it next to what people actually decided.
- "What if we had used the old rule?" is a guess.
- Past decisions cannot be compared fairly.
- A dispute comes down to memory.
The resultReal answers to "what if", on your own history.
What you get
Counterfactual replay, in practice
One request, two rules
See the old route and the new route side by side.
Signed answer
Keep the result as evidence of what the rule would have done.
Compared to people
Shows how the humans really decided.
How it works
Step by step
- 1
Choose a request
Any past one.
- 2
Choose a rule
Another version, or a draft.
- 3
Re-run
The policy engine decides again.
- 4
Signed result
Caught or not, with the real outcome.
Questions
About counterfactual replay
Does it touch the original decision?
No. The original stays as it was.
When is it useful?
After an incident, to show what would have stopped it.
Next
Works well with
Know what your agents did.
Start free. Guard your first agent in a few minutes, and keep a record anyone can check.
- Free plan, no card
- Signed, chained decisions
- Checked offline, without us
Signed decision
- Decision
- Approved, 2 of 2
- Action
- refund.issue, £540
- Rule
- Two reviewers must agree
- Sealed
- 14:09, key 7f2c
- Hash
- 5be2…7c44
Checked on your own machine
- Signature matches the public key
- Decision is in the log
- Every link before it adds up