The judge that learns
your codebase.
Judge & Jury reviews everything your coding agent ships and explains each finding with the evidence behind it. Then you tell it what it got wrong — and it retrains on that. The advice you reject stops coming back, and the mistakes you keep correcting turn into rules the agent gets before it writes a line.
STRIPE_SECRET_KEY is read at payments.py:14 but never set in the prod env. The first checkout request will crash.
“add new env vars to .env.example in the same commit”
One finding, one recalled prior, one preference it already learned — from three corrections you never had to repeat a fourth time.
Every judgment as a trace — what it found, what it cited, and whether you agreed.
How it learns
A reviewer that gets sharper, not noisier.
Most review tools are static: they make the same wrong call in month six that they made on day one, and you learn to ignore them. This one has an opinion you can correct — and correcting it is the whole interface.
It judges
Every run your agent makes, and every moment it claims done, gets reviewed. Findings carry a tier, a confidence, and the evidence behind them — never a bare opinion.
You label it
Confirm or reject, one click. That label is the entire training signal; there is no annotation queue and no separate labelling job to run.
It adapts
Classes you keep rejecting get demoted in recall, so noisy advice fades. Corrections that repeat in one category graduate into a preference injected before the agent writes code.
It retrains
Labelled findings become a corpus paired with the prompts that produced them. One command returns a tuned judge, and adopting it is a single line of config — not a migration.
What it learns from
You are already labelling. It just wasn't being kept.
Every time you push back on your agent, you produce a labelled example of what “wrong” means in your codebase. That signal is normally thrown away. Here it is the training set — no annotation queue, no separate labelling job, nothing extra to run.
Every finding you label
labelled advisoriesConfirmed and rejected advisories become paired training rows. Rejections matter as much as confirmations — they are what teach it to stop.
Every correction you type
labelled checkpointsWhen your agent says done and your next message pushes back, that is a labelled outcome. Three in one category graduate into a durable preference.
Every verdict it explains
resolved recordsEach blocked run is stored with its checks, its diff and its diagnosis, so a recurrence months later is recognisable rather than novel.
What changes as it learns
Noisy classes fade
A class you keep marking a false positive is demoted in recall, so it stops leading the evidence.
Corrections become rules
Three corrections in one category graduate into a candidate preference — and an active one is injected before the agent writes code.
Recall stops needing exact words
A recurrence phrased completely differently still matches the prior that explains it.
The model itself is retrained
Labelled findings pair with the prompts that produced them into a fine-tune corpus. Adopting the tuned judge is one line of config.
proofjury memory finetune
The vocabulary it judges in
Named failures, not vibes.
A judgment is only trainable if it is specific. Every finding is tagged with a class from an open taxonomy — which is what makes a recurrence recognisable months later, and what makes your label mean something the next time. Examples:
referenced in code, absent in the target env
the suite ran and something failed
the artifact never built
a live credential in tracked source
shipped without running the suite at all
dev config pointed at production
a required pre-deploy step never ran
schema change not applied to the target
manifest and lockfile disagree
a stub presented as done
the recorded QA run exited non-zero
QA configured, never executed
Why you can trust the label
Proof, not promises.
A judge you are meant to correct has to show its work, or your label is a coin flip. Every run writes a reproducible record, and the judge writes prose over that evidence rather than in place of it. Where a run is blocked, deterministic checks made that call — never the model.
- checks.json — every check, its result and its exit code
- diff.patch — exactly what was about to ship
- impact.json — the blast radius the checks measured
- context.json — the priors recalled, and why they matched
Bring your own key, scrubbed at the edge, and nothing uploads until you run proofjury connect.
Integrations
Exactly one of these can fail a run, and it does so from a recorded exit code — never from model output. The rest are evidence, context, or transport.
The one sponsor-backed check that can fail the gate — and it does so from a recorded exit code and a worktree digest, never from model output.
Finds the prior that matches a recurrence even when the wording is different. Priors are context the judge may cite — they can never short-circuit a verdict.
Authored policy the judge may cite. Cited statements carry a [source: doc] tag into the finding, so advice is traceable to a document a human wrote.
Routes each judge surface to a model and reports which one actually answered. It shapes how a verdict is explained, never what the verdict is.
Conventions it can cite
Policy your team authored. When the judge leans on one, the finding carries the source — so you can argue with the document rather than with the model.
Stop repeating yourself to your agent.
Correct it once. The judge keeps the correction, and the next agent on the repo inherits it.
proofjury connect