Verification, Critic Panels, and Confidence Gates

Place verifier nodes before fragile downstream synthesis. Use diverse lenses instead of identical critics. Keep unsupported claims out of final output.

~4hverificationcriticsllm-as-judge
Practice quiz (8 questions)Take final exam
Learning objectives
  • Place verifier nodes before fragile downstream synthesis.
  • Use diverse lenses instead of identical critics.
  • Keep unsupported claims out of final output.
Note

This course extends the existing Agentic Engineering track. The focus is not "more prompting"; it is shaping the work: loops, fan-out, fan-in, routing, verification, cost, and convergence.

Verification as graph structure

The graph gives you places to insert confidence gates: before a finding reaches synthesis, before code merges, before an answer is shown to a user.

Three verifier patterns

Use adversarial verify to refute findings, perspective-diverse verify for correctness/security/repro lenses, and judge panels to score multiple candidate outputs before synthesis.

const lenses = ['correctness', 'security', 'repro'];
const votes = await Promise.all(lenses.map(l => judge(finding, l)));
const keep = votes.filter(v => v.real).length >= 2;

Recap

A loop is a controlled cycle. A graph is the work shape around that cycle. The production skill is knowing which parts need model judgment, which parts are deterministic edges, and where confidence gates belong.

Verification, Critic Panels, and Confidence Gates — Exercises

Exercise 1 — Draw the hidden graph

Take a linear workflow you already use, such as "research sources → summarize → compare → write report". Draw every step as a node. Then remove every arrow where the downstream step does not consume upstream data.

Reference solution

A strong answer identifies independent nodes and labels edges by data, not by time. Example: "source A research" and "source B research" are independent; both feed a deterministic reduce step; the reduced evidence feeds synthesis.

Exercise 2 — Write node contracts

For three nodes in your graph, define:

  • input shape
  • output shape
  • failure mode
  • downstream consumer
Reference solution

Good contracts are narrow: { topic, sourceUrl } in; { claims: [{ text, sourceUrl, confidence }] } out. Avoid "return a useful summary" because the next node cannot validate it.

Exercise 3 — Add one safety gate

Place a verifier, schema validator, budget check, or human approval gate on the edge most likely to poison downstream work.

Reference solution

For a research graph, gate unsupported claims before synthesis. For a code graph, gate findings with reproduction or a second reviewer before reporting.