Case study — Deterministic verification over LLM-as-judge

Understand how the paper replaces an LLM judge with deterministic verifier oracles: the single promotion rule (a FAIL verdict backed by at least one evidence item), the three kinds of oracle, and the measured negative result that both supports and bounds the approach.

Track

Research

Module

5 · Research case studies

Prerequisites

Case study — Evidence-first agentic assessment (Module 5, lesson 1) · Comfort reading precision / recall as evaluation metrics

A verifier oracle is a function that, given the same inputs, returns the same result — no language-model call, no sampling, no network guessing. The paper’s central engineering claim is that every assessment family ends its work with such an oracle, and that the design question for each family is where, if anywhere, a model is allowed to act — with the invariant that a model is never the judge on the promotion path.

This reframes verification as re-derivation. A verifier does not ask "do you think this attack worked?"; it re-opens the source, re-runs a graph check, or re-reads a recorded exchange, and only a re-derived fact yields a FAIL. An accusation that cannot be re-derived stays UNKNOWN — refuting by elimination is treated as the same defect as promoting by default.

The paper reports that a result is promotable exactly when its verdict is FAIL and it carries at least one evidence item. There is one function that produces a finding, and it refuses unless both conditions hold and the evidence actually matches the hypothesis being promoted. No model call and no numeric score appear anywhere on this path.

the promotion predicate (as reported by the paper)
is_promotable(r)  ⇔  r.verdict == FAIL  ∧  |r.evidence| > 0

promote(r):
  reject unless is_promotable(r)
  reject unless r.evidence supports r.hypothesis   # exact match
  return Finding(...)          # the only function that mints a finding
# no LLM call, no score, no agent on this path

The paper describes seventeen assessment families and reports that they settle their proposals with one of three oracle kinds. The distinction is purely about where a model is permitted to help, never about who decides.

Deterministic oracle

The verifier re-derives the fact from the evidence — re-reading source, re-running a graph check, re-reading a recorded exchange. Only a re-derived fact yields FAIL; an unsupported accusation stays UNKNOWN. The paper reports thirteen of the seventeen families use this kind.

Hybrid oracle (deterministic authority)

A frontier model proposes candidates on a compact graph to widen recall, but a candidate is never a finding and is promoted only by deterministic re-derivation. The model holds no authority. The paper reports three families are hybrid in this sense.

Probe-based oracle

A stochastic probe (for example an injection probe) produces a report with its seed; a deterministic verifier re-reads that recorded report, so that given the report the verdict is deterministic. The paper reports one live family works this way.

Across all three, the shared contract is the same: given the same inputs the oracle returns the same result, with no LLM call and no sampling on the promotion path.

The paper reports that every deterministic verifier returns one of four values, and that the system deliberately keeps separate vocabularies for separate questions so that conflating them cannot blur which component is responsible.

the global verdict enum (as reported by the paper)
Verdict ∈ { PASS, FAIL, UNKNOWN, NOT_APPLICABLE }

FAIL            -> control violated / vulnerability confirmed (promotable)
PASS            -> control held                 (never promotes)
UNKNOWN         -> could not re-derive          (never promotes)
NOT_APPLICABLE  -> out of scope for this check  (never promotes)

The paper notes that this global Verdict is the only vocabulary that promotes. Other state machines in the system (a hypothesis-state ladder, a stateful agent verdict, a separate tribunal verdict for the adversarial lab) answer different questions and are kept distinct on purpose. Above promotion, a finding also carries a maturity derived from its proof levels — roughly RAW, CONFIRMED (a deterministic verifier settled it) and VALIDATED (reproduced by a replay) — which is again a function of the proofs, with no model involved.

The paper’s audit of a documental-AI target makes the distinction concrete. The agents produced a large pool of candidate findings about a poisonable store that were all of a synthetic adjacency class: a verifier could only "confirm" them by re-running the very query that produced the belief. Those are circular and do not carry a real dataflow path, so they stay UNKNOWN. A separate, much smaller pool of genuinely re-derived findings carried real def-use dataflow paths and survived.

the decomposition the paper reports (illustrative shape)
candidate pool (large)   -> circular adjacency class  -> UNKNOWN (filtered)
re-derived pool (small)  -> real def-use dataflow      -> CONFIRMED findings

paper’s point: "N candidates survived deterministic re-derivation"
is more informative than any raw count of flagged candidates.

The most scientifically interesting result in the paper runs against intuition, and the paper reports it precisely because it is negative. When the deterministic verifiers were taken off the familiar target and run over a held-out corpus of external repositories, recall degraded gracefully but precision collapsed: the paper reports micro-recall falling from roughly 0.339 to 0.196 while micro-precision fell from about 0.497 to 0.102, with false positives per repository rising from around 9 to over 50. A few detectors that over-fire off-target (a retrieval-scoping detector, a hardcoded-secrets detector and an authorization detector) dominate the loss.

The paper argues this cuts both ways. It supports the thesis: without a strict promotion gate that leaves un-re-derived accusations as UNKNOWN, an off-target run would flood the user with false findings — the precision collapse is what the gate is designed to contain. And it bounds the thesis honestly: the deterministic oracles are not target-independent, and an off-target micro-precision near 0.10 is a real limitation, not a footnote.

• ▸Verifier correctness is assumed, not proven (LIMITATION): the oracle guarantees re-derivation happened, not that the re-derivation logic is right.
• ▸Precision is not target-independent (MEASURED LIMITATION): a handful of over-firing detectors cause the off-target collapse; the in-domain precision was partly inflated by the familiar corpus lacking negative traps.
• ▸No head-to-head benchmark (OPEN): the paper does not report a true/false-positive comparison against existing red-team tools; that comparison is specified as future work requiring a shared labelled target.
• ▸Zero-model-on-the-path is tested for the deterministic path (TESTED): the paper reports tests asserting zero model calls and zero cost on the promotion and verification path, plus an import barrier forbidding any LLM client in the verification layer — but the hybrid families still invoke a frontier model in the propose role, which those tests do not cover.

Expected direction: Accurate version: the model proposed 500 candidates (widening recall); a deterministic verifier re-derived 12 as CONFIRMED findings, and the rest were not promoted (UNKNOWN unless re-derived). The model held no authority. The off-target result matters because precision collapses on unfamiliar targets (micro-precision ~0.10, ~50 false positives per repo as the paper reports); without the FAIL-plus-evidence gate leaving un-re-derived candidates as UNKNOWN, those 488 unconfirmed proposals could be reported as findings and swamp the user.

State the promotion rule exactly.

A result is promotable iff its verdict is FAIL and it carries at least one evidence item that matches the hypothesis; only then does the single promote() function mint a finding.

What is the difference between a hybrid oracle and a probe-based oracle?

In a hybrid oracle a frontier model proposes candidates on a graph but promotion is by deterministic re-derivation; in a probe-based oracle a stochastic probe emits a seeded report and a deterministic verifier re-reads that recorded report. Neither lets a model decide.

Why does the paper report the off-target precision collapse rather than hide it?

Because it both supports the thesis (a strict gate is needed precisely to contain a flood of false findings off-target) and bounds it honestly (the oracles are not target-independent; off-target precision near 0.10 is a real limitation).

Is "zero model calls on the promotion path" a design claim or a tested one, and does it cover the whole system?

It is TESTED (tests assert zero LLM calls and zero cost on the deterministic promotion/verification path, with an import barrier). It does not cover the hybrid families’ propose role, where a frontier model is still invoked.