Observe
Know if a change made your agent worse, before it ships
Run test sets against your agent, compare runs with honest statistics, and block a release when it gets worse.
Pass rate
Baseline against this runGate
- Rule
- No drop over 3 points
- Drop
- −11 points
- Confidence
- 95% interval
- Result
- Release blocked
In CI
$ xevrion evals gate run_41FAILED. pass rate fell 90% → 79%exit 1Why it matters
What evals and release gate changes
- Run test sets against your agent.
- Compare runs with honest statistics.
- Block a release when it gets worse.
- A new prompt ships on a hunch.
- One run gives one opinion.
- Regressions reach your customers first.
The resultOnly ship changes that are better.
What you get
Evals and release gate, in practice
Honest numbers
Every score has a range. A small test set says so, instead of showing a confident number.
Judge, with a check
An AI judge is only trusted once it agrees with your people often enough on enough cases.
A gate for CI
Fail the build when a run is worse than the one you trust.
How it works
Step by step
- 1
Build a test set
From real reviews, or by hand.
- 2
Run it
On each version of your agent.
- 3
Compare
Same cases, side by side.
- 4
Gate the release
Pass, or stop with the reason.
Questions
About evals and release gate
Why would a judge not be trusted?
If it has been checked on fewer than 30 cases, or disagrees with your reviewers too often.
Can I run it in CI?
Yes. The SDK and API return a pass or fail you can use in any pipeline.
Next
Works well with
Know what your agents did.
Start free. Guard your first agent in a few minutes, and keep a record anyone can check.
- Free plan, no card
- Signed, chained decisions
- Checked offline, without us
Signed decision
- Decision
- Approved, 2 of 2
- Action
- refund.issue, £540
- Rule
- Two reviewers must agree
- Sealed
- 14:09, key 7f2c
- Hash
- 5be2…7c44
Checked on your own machine
- Signature matches the public key
- Decision is in the log
- Every link before it adds up