Peer Elicitation Game

A generator produces outputs that are evaluated by a panel of discriminator agents from distinct base models; discriminator utilities are computed via a mutual-information score that provably incentivises truthful reporting without requiring ground-truth labels, and the system converges to a truthful Nash equilibrium.

Properties

family: Evaluation & Ranking
maturity: frontier
diagram: sealed
whyMultiAgent: Truthful peer prediction only works when evaluators cannot collude or share signals before committing scores; multiple heterogeneous discriminators are structurally required so that each agent's best response is honest evaluation, a property that collapses with a single judge.
mast: [Cat 2][Cat 3]
useCases: [Factual accuracy evaluation without labelled data][Hallucination detection at inference time with no fine-tuning][Alignment verification for frontier models where human labels are scarce]
source: Peer Elicitation Games · arXiv:2505.13636

Provenance

Source: https://arxiv.org/abs/2505.13636

Peer Elicitation Game

Properties

family: Evaluation & Ranking
maturity: frontier
diagram: sealed
whyMultiAgent: Truthful peer prediction only works when evaluators cannot collude or share signals before committing scores; multiple heterogeneous discriminators are structurally required so that each agent's best response is honest evaluation, a property that collapses with a single judge.
mast: [Cat 2][Cat 3]
useCases: [Factual accuracy evaluation without labelled data][Hallucination detection at inference time with no fine-tuning][Alignment verification for frontier models where human labels are scarce]
source: Peer Elicitation Games · arXiv:2505.13636

Provenance

Source: https://arxiv.org/abs/2505.13636