N independent validator agents evaluate the same output; the result is accepted only when M validators agree. Disagreement triggers retry or escalation. Correctness improves with N at the cost of N× validation compute.
A medical AI platform generates treatment recommendations for clinical decision support. A single AI recommendation carries a 12% error rate on rare conditions. Five independent validator models - each fine-tuned on a different clinical subspecialty - evaluate each recommendation. If 3 or more of 5 agree: publish with confidence score. If fewer than 3 agree: trigger regeneration with updated context. After 3 failed attempts: escalate to a human physician. Result: 2.3% error rate on rare conditions, 0.8% escalation rate to humans.
The key insight: error reduction is not from averaging five opinions - it is from requiring distributed agreement. A recommendation that only two subspecialists accept is flagged as uncertain regardless of how confident those two validators are. The threshold M is the governance parameter: raising it reduces error rate at the cost of escalation rate. The platform sets M=3 as a clinical policy decision, not a technical one.
| Metric | Signal |
|---|---|
| Consensus rate (first attempt) | % of cases reaching threshold M on the first generation - primary throughput signal. |
| Mean attempts per decision | Average retry cycles before consensus or escalation - drives total compute cost per decision. |
| Inter-validator kappa | Agreement rate between validator pairs on identical inputs - measures validator independence. Low kappa = correlated validators. |
| Escalation rate | % of cases escalated to human after max retries - primary human-load signal; must stay within capacity. |
| Quality vs human baseline | Consensus decision accuracy vs physician decision on identical cases - validates that the threshold M is set correctly. |
| Node | What it does | What it receives | What it produces |
|---|---|---|---|
| Response Generator | Produces a treatment recommendation from the clinical query; re-runs with disagreement context on retry | Clinical query (+ disagreement context on retry) | Treatment recommendation candidate |
| Validator Fan-out | AND-splits the candidate to all 5 validators in parallel | Recommendation candidate | Candidate copy dispatched to each validator lane |
| Validators 1..5 | Each independently evaluates the candidate against its subspecialty knowledge; returns accept/reject with rationale | Recommendation candidate | Accept/reject verdict + rationale per validator |
| Vote Tally | AND-join: counts accept votes, checks against threshold M=3, routes to pass or fail path | 5 validator verdicts | Vote count + threshold result (pass / fail) |
| Publish Response | Releases the accepted recommendation to the clinical system with confidence score | Accepted recommendation + vote count | Published recommendation with confidence |
| Regenerate | Produces a new candidate incorporating validator disagreement context; routes back to Generator | Failed candidate + disagreement rationale | Revised candidate for next validation round |
| Human Escalation | Routes the case to a physician after 3 failed consensus attempts | All failed candidates + validator rationale | Escalation packet for physician review |
| Origin of Value | Where it appears | How it is captured |
|---|---|---|
| Future Cashflow | Recommendation accuracy on rare conditions | Error rate reduction from 12% to 2.3% is the primary value event. This is only realized when validators are independently grounded - correlated validators produce the same error reduction at 1× compute. |
| Governance | Threshold M and escalation policy | M=3 is a clinical policy decision encoded as a governance parameter. Changing M from 3 to 4 changes the platform's risk tolerance without altering model weights. The threshold is the organization's primary governance lever over AI output quality. |
| Risk Exposure | Correlated validator failure | If all 5 validators share a blind spot (e.g., all fine-tuned on the same clinical corpus), consensus is achieved on wrong answers. The pattern produces 5× false confidence. Validator independence is the critical risk variable - measure inter-validator disagreement rate, not just consensus rate. |
| Conditional Action | N× validation compute per decision | Unlike Competitive Evaluation where compute is front-loaded in generators, Consensus compute is in validators. On retry cycles, total compute is (N validators) × (attempts). Escalation rate and mean attempts are the cost drivers, not N alone. |
VCM analog: Consensus Token. Value is released only when M validators co-sign. This is the exact structure of a blockchain commit - distributed agreement before state change. The threshold M is the governance parameter. Increasing M raises security (fewer false positives) at the cost of throughput (higher escalation rate). The escalation path is the safety valve when the consensus mechanism deadlocks.
All 5 validators are fine-tuned on overlapping clinical datasets. On a novel drug interaction not represented in the training corpus, all 5 accept an incorrect recommendation with high confidence. Consensus is 5-of-5 - maximum confidence signal. The error passes through undetected. The platform's error rate on out-of-distribution cases is indistinguishable from single-agent performance, but confidence scores are artificially inflated. Fix: validator independence must be enforced architecturally - different fine-tuning corpora, different base models, or different knowledge retrieval backends per validator. Measure inter-validator disagreement rate on a held-out test set as a proxy for independence.
Validators are trained with an implicit bias toward agreement (e.g., RLHF reward for accepting rather than rejecting). Over time, validators become lenient to avoid generating disagreement signals. The threshold M=3 becomes effectively M=1 in practice - all 5 validators accept most candidates. Escalation rate drops to near zero, masking a rising error rate. Fix: audit validator rejection rate per category. A validator with less than 5% rejection rate on any category is not functioning as a validator. Calibrate validators against held-out adversarial cases with known errors.
Threshold M is set to 4-of-5. Validators are well-calibrated but strict. On 40% of cases, fewer than 4 validators agree on the first attempt. After 3 retry rounds, 15% of cases escalate to human physicians. Human review capacity is sized for 1% escalation. The escalation queue backs up, defeating the automation goal. Fix: model escalation rate before deploying M. Run the system on a historical case sample to measure empirical consensus rate at each threshold. Set M to achieve the target escalation rate, not the target error rate in isolation.
A regenerated candidate on retry 2 is structurally similar to the failed candidate from retry 1 because the Generator does not have sufficient signal in the disagreement context to produce a meaningfully different recommendation. All three attempts fail, escalation triggers. But the escalation packet contains 3 nearly identical failed candidates - the physician cannot determine which aspect caused repeated failure. Fix: the Regenerate node must receive structured disagreement rationale, not just a rejection flag. Each retry must produce a measurably different candidate - measure inter-attempt similarity (e.g., semantic similarity score) and halt the retry loop if similarity exceeds a threshold.
| Variant | Modification | When to use |
|---|---|---|
| Weighted Consensus | Validators have different weights based on subspecialty relevance to the query; M threshold is a weighted sum, not a head count | Validator expertise is not uniform across query types; a cardiology validator should count more on a cardiac case than a dermatology one |
| Adversarial Consensus | One validator is a red-team agent whose job is to find flaws; acceptance requires N-1 validators to accept AND the red-team agent to fail to find a critical flaw | High-stakes outputs where the risk of false positives (accepting bad output) exceeds the cost of false negatives (escalating good output) |
| Sequential Consensus | Validators run serially; each sees the prior validators' verdicts before issuing its own; stops early if M is reached before all N run | Compute budget is tight; early stopping reduces expected validation cost; validator independence is acceptable to sacrifice for cost reduction |
| Pattern | Relationship |
|---|---|
| 10.15 Evaluator-Optimizer | Single critic loop vs distributed consensus - Evaluator-Optimizer improves one output through sequential critique; Consensus requires simultaneous agreement from N independent validators |
| 20.22 Human-in-the-Loop | Consensus provides the escalation trigger - when distributed AI consensus fails after max retries, the Human-in-the-Loop pattern handles the escalation path |
| 20.24 Competitive Evaluation | Selection among candidates vs agreement on one candidate - Competitive Evaluation picks the best of N outputs; Consensus accepts one output only when N validators agree on it |
The threshold M and the validator independence architecture are the organizational IP. Validators are individually commodity - any fine-tuned model that produces accurate verdicts qualifies. The governance layer (M, escalation policy, retry limit) encodes the organization's risk tolerance as machine-executable policy. This is the asset: not the models, but the policy parameters and the evidence base that validates them.
A medical platform that has calibrated M=3 against 50,000 historical cases with known outcomes has an evidence-backed governance parameter. A competitor using the same architecture with M chosen by intuition is carrying unknown error rates. The calibration dataset is the moat, not the model.
Red flag: a Consensus system with no per-validator performance tracking cannot detect vote inflation. If all validators are treated as equivalent black boxes with no individual accuracy monitoring, the governance layer degrades silently over time. Validator drift - one validator becoming systematically more lenient - is invisible without per-validator metrics. A system that only reports aggregate consensus rate cannot be audited at the validator level, making due diligence on safety claims impossible.