---
title: "LLM as a judge: biases, validation, and when to distrust it"
type: "deep dive"
url: "https://aifordevs.org/articles/llm-as-a-judge"
markdown: "https://aifordevs.org/articles/llm-as-a-judge.md"
publisher: "AI for Devs"
license: "https://aifordevs.org/reuse"
attribution: "Credit AI for Devs and link to https://aifordevs.org/articles/llm-as-a-judge"
description: "LLM as a judge means grading model output with a model. See what Zheng et al. measured, where bias enters, and how to validate a judge against human labels."
author: "AI for Devs"
published: "2026-11-08"
level: "Advanced"
topics: ["evals"]
---

# LLM as a judge: biases, validation, and when to distrust it

LLM as a judge means grading model output with a model. See what Zheng et al. measured, where bias enters, and how to validate a judge against human labels.

> **TL;DR**
>
> - LLM as a judge means using one model to grade another model's output against a written criterion or by comparing two answers.
> - Zheng et al. (2023) reported 85 percent agreement between GPT-4 and humans on MT-Bench with ties excluded, and 66 percent with ties and inconsistent votes counted. Read both numbers.
> - Position, verbosity, and self-preference biases all have primary measurements, and each has a test you can run on your own judge.
> - Validate against human labels with kappa and a bootstrap interval. In this article's illustrative dataset, 88 percent agreement hides a kappa of 0.61 (95 percent interval 0.46 to 0.74) and a 35 percent false-pass rate. Python 3.15.1 ran this with scripted clients and no hosted model.
> - Do not trust a judge alone on factual claims without a reference, on text an attacker can write, or when the lower end of its kappa interval is under 0.60.

LLM as a judge is the practice of using a language model to grade another model's output against a written criterion or by comparing answers. It lets you score open-ended outputs at a volume that human reviewers cannot reach. A judge can also agree with humans 88 percent of the time and still pass a third of the outputs a person would reject. This article starts from the 2023 MT-Bench paper, reads the later bias work, and shows the calculation to run before you trust the score.

## What LLM as a judge means in practice

LLM as a judge turns a quality question into a model call that returns a verdict. Three forms are common: grading one answer, comparing two answers, and grading against a reference answer. Each form has different failure modes, so the choice of form is part of the measurement.

[Zheng et al. (2023)](https://arxiv.org/abs/2306.05685) tested pairwise comparison and single-answer grading. [OpenAI's evaluation guide](https://developers.openai.com/api/docs/guides/evaluation-best-practices) lists pairwise comparison, single-answer grading, and reference-guided grading, and recommends pairwise or pass/fail formats for reliability.

| Form | Input | Output | Best fit | Main weakness |
| --- | --- | --- | --- | --- |
| Single answer | Question, answer, criterion | Pass or fail, or a score | Regression tracking on one model | Absolute scoring is easiest to attack |
| Pairwise | Question, two answers | Winner or tie | Choosing between two versions | Position bias |
| Reference-guided | Question, answer, known-correct reference | Pass or fail | Math, facts, extraction | Needs a trustworthy reference |

A judge fits inside a larger [eval](https://aifordevs.org/glossary#eval) suite, usually as the metric for open-ended answers. The [LLM evaluation guide](https://aifordevs.org/articles/llm-evaluation) covers where it sits, and the [glossary entry for LLM as a judge](https://aifordevs.org/glossary#llm-as-a-judge) gives the short definition.

## What Zheng et al. measured

The 2023 MT-Bench paper reported that GPT-4 agreed with human preferences more often than humans agreed with each other, in one specific setup. That setup excludes ties. The paper reports both ways of counting, and the gap between them is the first thing to read about judge accuracy.

MT-Bench has 80 multi-turn questions in 8 categories, and the paper compares six models. About 3,000 votes came from 58 expert labelers, mostly graduate students.

Setup S1 counts ties and position-inconsistent votes as ties, and its chance baseline is 33 percent. Setup S2 counts only non-tie votes, and its chance baseline is 50 percent.

| Comparison, first turn | S1 (ties and inconsistent counted) | S2 (non-tie only) |
| --- | --- | --- |
| GPT-4 pairwise vs humans | 66% (1,343 votes) | 85% (859 votes) |
| GPT-4 single answer vs humans | 60% (1,280 votes) | 85% (739 votes) |
| Human vs human | 63% (721 votes) | 81% (479 votes) |

The 85 percent headline holds only after dropping every tie and every case where the judge changed its verdict when the answers swapped. The same table shows S2 removing more than a third of the GPT-4 pairwise votes, from 1,343 to 859. Your own validation has to make that choice about what counts in the open.

Treat this as a 2023 result. The authors used `gpt-4-0314`, and they say the examples may not be fully reproducible on later GPT-4 versions. They also say the work largely sets aside safety, including honesty and harmlessness.

## Position bias

Position bias means a judge's verdict depends on which slot an answer occupies, so swapping the order changes the winner. On MT-Bench, GPT-4 stayed consistent in 65.0 percent of swapped cases with the default prompt, against 46.2 percent for GPT-3.5 and 23.8 percent for Claude-v1. Run each pair in both orders.

**Figure 1.** A consistent judge picks the same answer in both orders. A position-biased judge picks the same slot, so the winner changes when the order does, and the pair should count as a tie.

Few-shot examples help and cost you. The MT-Bench authors raised GPT-4 consistency from 65.0 to 77.5 percent with few-shot examples, and longer prompts made API calls about 4 times as expensive. Check current rates in [Claude API pricing](https://aifordevs.org/articles/claude-api-pricing) or [Gemini API pricing](https://aifordevs.org/articles/gemini-api-pricing) before you run a few-shot judge on every trace.

[Wang et al. (2023)](https://arxiv.org/abs/2305.17926) showed how far order alone can move a ranking. Vicuna-13B beat ChatGPT on 66 of 80 queries with ChatGPT as the evaluator, and their calibration methods aim to reduce exactly that.

[Shi et al. (2024)](https://arxiv.org/abs/2406.07791) tested 15 judges across more than 150,000 evaluation instances. Position bias varied by judge and task, and it was strongly affected by the quality gap between solutions.

Close versions of your prompt are the hard case, because the authors found answers of similar quality are harder to judge and more prone to position bias. Count order-inconsistent pairs as ties, and report the tie rate next to the win rate.

## Verbosity bias

Verbosity bias means a judge prefers the longer answer when the content is otherwise equal or worse. An MT-Bench test padded 23 answers with a rephrased list. Claude-v1 and GPT-3.5 preferred the padded version 91.3 percent of the time, and GPT-4 did so 8.7 percent of the time.

[Saito et al. (2023)](https://arxiv.org/abs/2310.10076) defined a verbosity metric where positive values favor wordier answers, and reported 0.328 for GPT-4 and 0.428 for GPT-3.5. Once the word-count difference grows large enough, GPT-4 almost always preferred the longer answer in their study.

Length also distorts leaderboards like the ones behind [best LLM for coding](https://aifordevs.org/articles/best-llm-for-coding) rankings. [Dubois et al. (2024)](https://arxiv.org/abs/2404.04475) estimated AlpacaEval preferences at equal output length and raised its Spearman correlation with Chatbot Arena from 0.94 to 0.98.

A prompt change that adds a closing paragraph can look like a win for that reason alone.

Test yours directly. Take a passing answer, add a paragraph restating it, and re-judge. If the verdict flips, or the pairwise judge prefers the padded version, you have measured the bias on your own rubric.

## Self-preference bias

Self-preference bias means a judge scores text from its own model family higher than a neutral rater would. The MT-Bench authors said their data could not settle the question. Later studies found the bias in GPT-4 and did not find it uniformly across models, so use a judge from a different family.

[Panickssery et al. (2024)](https://arxiv.org/abs/2404.13076) measured summary preferences on XSUM and CNN. GPT-4's normalized preference for its own summaries was 0.705 and 0.912, against 0.511 and 0.505 for Llama-2-7b, where 0.5 would be indifference.

GPT-3.5 scored 0.582 and 0.431, so the direction is not universal. After fine-tuning, self-recognition and self-preference moved together in a linear relationship.

[Wataoka et al. (2024)](https://arxiv.org/abs/2410.21819) reported significant self-preference in GPT-4 and traced it to perplexity, since judges gave higher scores to lower-perplexity text than humans did. That points to a broader risk: a judge may reward text that resembles what its training made likely.

Anthropic's [eval docs](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests) note in their judge examples that it is generally best to evaluate with a different model from the one that generated the output. When you compare model families, run the judge from a third family, or run two judges and report where they disagree.

## How to build a judge you can check

Build a judge as a small function with one criterion, a binary verdict, and an explicit error state. One criterion per judge makes disagreement diagnosable, and a binary verdict is easier to validate than a 1 to 5 score. Require reasoning before the verdict, which OpenAI's guide says improves eval performance.

The MT-Bench paper shows why the format matters. On 10 math questions with 20 test cases, the default prompt failed 14 of 20, chain-of-thought failed 6 of 20, and a reference-guided prompt failed 3 of 20. Give the judge the correct answer whenever you have one.

Keep these rules for the first version:

1. Write one judge per criterion, in the language of your rubric, and keep it in its own [system prompt](https://aifordevs.org/glossary#system-prompt).
2. Return [JSON](https://aifordevs.org/glossary#structured-outputs) with reasoning first and the verdict last.
3. Treat any unparseable or out-of-range output as an error, never as a pass.
4. Add few-shot examples only from your development set.
5. Pin the judge model and version, and log the raw output.

The code below implements those rules plus the swap test, against a scripted client so it runs offline. Replace `scripted_client` with a function that calls your provider.

```python title="judge.py"
#!/usr/bin/env python3
"""A pass/fail judge and a position-swap check, run against a scripted client."""
import json

CRITERION = "The answer states that refunds over $500 need a second approver."

SINGLE_PROMPT = """Grade one answer against one criterion.
Question: {question}
Answer: {answer}
Criterion: {criterion}
Return JSON only: {{"reasoning": "<one sentence>", "verdict": "pass" or "fail"}}"""

PAIR_PROMPT = """Which response better meets the criterion?
Question: {question}
Response A: {a}
Response B: {b}
Criterion: {criterion}
Return JSON only: {{"reasoning": "<one sentence>", "winner": "A" or "B" or "tie"}}"""


def judge_single(client, question, answer, criterion=CRITERION):
    raw = client(SINGLE_PROMPT.format(question=question, answer=answer, criterion=criterion))
    data = json.loads(raw)
    if data.get("verdict") not in {"pass", "fail"}:
        return {"verdict": "error", "reasoning": raw}
    return data


def judge_pair_swapped(client, question, x, y, criterion=CRITERION):
    """Ask twice with the order swapped. Disagreement becomes a tie."""
    first = json.loads(client(PAIR_PROMPT.format(question=question, a=x, b=y, criterion=criterion)))["winner"]
    second = json.loads(client(PAIR_PROMPT.format(question=question, a=y, b=x, criterion=criterion)))["winner"]
    first_pick = {"A": "x", "B": "y"}.get(first, "tie")
    second_pick = {"A": "y", "B": "x"}.get(second, "tie")
    if first_pick == second_pick:
        return first_pick, first, second
    return "tie (inconsistent)", first, second


def scripted_client(replies):
    queue = iter(replies)
    return lambda _prompt: next(queue)


question = "Who approves a $600 refund?"
good = "Refunds over $500 need a second approver in finance."
bad = "Refunds over $500 are approved by the finance team."

client = scripted_client([
    '{"reasoning": "It names a second approver.", "verdict": "pass"}',
    '{"reasoning": "No second approver is named.", "verdict": "fail"}',
    '{"reasoning": "Unparseable label.", "verdict": "maybe"}',
])
print("single:", judge_single(client, question, good)["verdict"])
print("single:", judge_single(client, question, bad)["verdict"])
print("single:", judge_single(client, question, good)["verdict"])

# Position-biased client: always prefers the first response.
biased = scripted_client(['{"reasoning": "First.", "winner": "A"}'] * 2)
print("biased pair:", judge_pair_swapped(biased, question, good, bad))

# Consistent client: prefers the same response in both orders.
stable = scripted_client(['{"reasoning": "x wins.", "winner": "A"}', '{"reasoning": "x wins.", "winner": "B"}'])
print("stable pair:", judge_pair_swapped(stable, question, good, bad))
```

Tested with Python 3.15.1 on November 8, 2026:

```text
single: pass
single: fail
single: error
biased pair: ('tie (inconsistent)', 'A', 'A')
stable pair: ('x', 'A', 'B')
```

The third line shows an out-of-range verdict becoming an error. The biased pair shows a client that always prefers slot A being counted as a tie, which is the logic of Figure 1.

This run exercised prompt formatting, JSON parsing, the error state, and swap logic. It did not exercise any model's actual judgment.

## Validate the judge against human labels

Validate a judge by scoring a sealed set of human-labeled outputs once, then reporting agreement, a chance-corrected statistic, and a false-pass rate with confidence intervals. Raw agreement alone overstates the judge when one label dominates, so a tool that always says pass can look accurate on a mostly passing set.

**Figure 2.** Human labels split into a development set that can change the rubric and a holdout that cannot. The holdout's kappa interval decides how far the judge is trusted.

Sample real outputs into a [golden dataset](https://aifordevs.org/glossary#golden-dataset) and oversample failures on purpose, so the false-pass rate has enough failed cases to estimate. Have two people label an overlapping subset first, because the agreement between humans is the ceiling for any judge on that rubric.

If two people cannot agree, fix the rubric before you touch a model.

Split the labels and seal the holdout. Figure 2 shows a 30 and 70 percent split, which is a starting point. Tune the rubric and examples only on the development set, and score the holdout once at the end, as OpenAI's guide says to [calibrate automated scoring against human feedback](https://developers.openai.com/api/docs/guides/evaluation-best-practices).

The script below computes raw agreement, Cohen's kappa, and a percentile bootstrap interval. [Cohen's kappa](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.cohen_kappa_score.html) is `(p_o - p_e) / (1 - p_e)`, where `p_o` is observed agreement and `p_e` is the agreement expected by chance from each rater's label rates.

The label counts below are illustrative, written by hand for this article. They are not a measurement of any model.

```python title="agreement.py"
#!/usr/bin/env python3
"""Judge versus human labels: agreement, kappa, and a bootstrap interval.
The label counts below are ILLUSTRATIVE, written by hand for this article.
They are not a measurement of any model or any judge."""
import math
import random

SEED, RESAMPLES = 42, 10_000
Z = 1.959964  # two-sided 95 percent


def expand(tp, fn, fp, tn):
    """(human, judge) pairs, 1 for pass and 0 for fail."""
    return [(1, 1)] * tp + [(1, 0)] * fn + [(0, 1)] * fp + [(0, 0)] * tn


def cohen_kappa(pairs):
    n = len(pairs)
    observed = sum(h == j for h, j in pairs) / n
    h_pass = sum(h for h, _ in pairs) / n
    j_pass = sum(j for _, j in pairs) / n
    expected = h_pass * j_pass + (1 - h_pass) * (1 - j_pass)
    if expected == 1:
        return 1.0
    return (observed - expected) / (1 - expected)


def agreement(pairs):
    return sum(h == j for h, j in pairs) / len(pairs)


def false_pass_rate(pairs):
    fails = [j for h, j in pairs if h == 0]
    return sum(fails) / len(fails) if fails else float("nan")


def percentile(values, q):
    ordered = sorted(values)
    return ordered[int(q * (len(ordered) - 1))]


def bootstrap(pairs, stat, rng):
    n = len(pairs)
    samples = [stat(rng.choices(pairs, k=n)) for _ in range(RESAMPLES)]
    low, high = percentile(samples, 0.025), percentile(samples, 0.975)
    return f"[{low:.3f}, {high:.3f}]"


def report(name, pairs):
    rng = random.Random(SEED)
    n = len(pairs)
    print(f"{name} (n={n})")
    print(f"  raw agreement    {agreement(pairs):.3f}  95% CI {bootstrap(pairs, agreement, rng)}")
    print(f"  Cohen's kappa    {cohen_kappa(pairs):.3f}  95% CI {bootstrap(pairs, cohen_kappa, rng)}")
    print(f"  false-pass rate  {false_pass_rate(pairs):.3f}  95% CI {bootstrap(pairs, false_pass_rate, rng)}")


# Lenient judge on a set where humans fail 40 of 200 outputs.
lenient = expand(tp=150, fn=10, fp=14, tn=26)
report("lenient judge, 20% human-fail rate", lenient)

# Same raw agreement, rarer failures: kappa collapses.
skewed = expand(tp=180, fn=10, fp=10, tn=0)
report("skewed set, 5% human-fail rate", skewed)

# Correct the judge's pass rate on unlabeled production data.
sens = 150 / 160  # P(judge pass | human pass)
spec = 26 / 40  # P(judge fail | human fail)
for observed in (0.70, 0.82, 0.90):
    corrected = (observed + spec - 1) / (spec + sens - 1)
    print(f"judge pass rate {observed:.2f} -> corrected true pass rate {corrected:.3f}")

# How wide is a false-pass estimate of 0.35 at different failure counts?
def wilson(p, n, z=Z):
    centre = (p + z * z / (2 * n)) / (1 + z * z / n)
    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
    return centre - half, centre + half


for fails in (20, 40, 100, 200, 400):
    low, high = wilson(0.35, fails)
    print(f"human-failed cases {fails:>3}: false-pass 0.35, 95% CI [{low:.3f}, {high:.3f}], width {high - low:.3f}")
```

Run with Python 3.15.1, standard library only. The kappa values match scikit-learn 1.9.1's `cohen_kappa_score` (0.6104 and minus 0.0526).

```text
lenient judge, 20% human-fail rate (n=200)
  raw agreement    0.880  95% CI [0.835, 0.925]
  Cohen's kappa    0.610  95% CI [0.458, 0.743]
  false-pass rate  0.350  95% CI [0.206, 0.500]
skewed set, 5% human-fail rate (n=200)
  raw agreement    0.900  95% CI [0.855, 0.940]
  Cohen's kappa    -0.053  95% CI [-0.075, -0.026]
  false-pass rate  1.000  95% CI [1.000, 1.000]
judge pass rate 0.70 -> corrected true pass rate 0.596
judge pass rate 0.82 -> corrected true pass rate 0.800
judge pass rate 0.90 -> corrected true pass rate 0.936
human-failed cases  20: false-pass 0.35, 95% CI [0.181, 0.567], width 0.386
human-failed cases  40: false-pass 0.35, 95% CI [0.221, 0.505], width 0.284
human-failed cases 100: false-pass 0.35, 95% CI [0.264, 0.447], width 0.184
human-failed cases 200: false-pass 0.35, 95% CI [0.287, 0.418], width 0.131
human-failed cases 400: false-pass 0.35, 95% CI [0.305, 0.398], width 0.093
```

**Figure 3.** Similar raw agreement, opposite conclusions. On the illustrative counts, 88 percent agreement carries a kappa of 0.61, and 90 percent agreement on rare failures carries a kappa of minus 0.05.

Start with the left panel. An 88 percent agreement rate sounds fine until you see a kappa of 0.61 and a judge that passes 14 of the 40 outputs a person failed. The bootstrap interval for that kappa runs from 0.458 to 0.743, so this data cannot rule out a weak judge.

The bootstrap resamples the label pairs with replacement, recomputes the statistic each time, and reads the interval off the distribution. [SciPy's docs](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bootstrap.html) describe the percentile method and note that it is rarely used in practice. It appears here for clarity, and bias-corrected methods exist for small or skewed data.

Label enough failures, because the false-pass rate is only as precise as your failed-case count. At a 0.35 false-pass rate, 40 failed cases give a 95 percent interval from 0.221 to 0.505, and 400 give 0.305 to 0.398. A 200-row sample with 20 percent failures is enough to find a weak judge and too small to certify a strong one.

## Correct the pass rate with the judge's error rates

A judge's raw pass rate on unlabeled traffic is a biased estimate of the true pass rate unless the judge is perfect. [Lee et al. (2025)](https://arxiv.org/abs/2511.21140) give a correction that uses the judge's sensitivity and specificity from your human-labeled calibration set. On the illustrative labels, a measured pass rate of 0.90 corresponds to 0.936.

The estimator is `theta = (p + q0 - 1) / (q0 + q1 - 1)`, where `p` is the judge's observed pass rate. `q1` is sensitivity, the share of human-passed outputs the judge passes. `q0` is specificity, the share of human-failed outputs the judge fails.

The illustrative judge has a sensitivity of 150/160 and a specificity of 26/40. Its output above shows an imperfect judge compressing the range. A measured pass rate of 0.70 maps to a true rate of about 0.596, and 0.90 maps to about 0.936.

Bad weeks look better than they were and good weeks look worse, so trend lines on raw judge scores understate real change.

Lee et al. also build confidence intervals that account for uncertainty in both the test set and the calibration set, which the point estimate above does not do. If you report a corrected number, report its interval too, using their method.

## When a judge should not be trusted

Do not let a judge alone gate a change if code can check the output or an attacker can shape the text. Its validation must also clear a kappa lower bound of 0.60. A judge is a measurement instrument, and an unvalidated one produces numbers that look more precise than they are.

| Situation | Evidence | What to do instead |
| --- | --- | --- |
| Facts or math with no reference | Zheng et al.: default prompt failed 14 of 20 math cases | Supply a reference, compute the answer in code, or use retrieval to limit [hallucination](https://aifordevs.org/glossary#hallucination) |
| Close calls between similar versions | Shi et al.: position bias strongly tied to the quality gap | Swap orders, count ties, add human review |
| Text an attacker can influence | Raina et al., Shi et al. | Code checks and human review for adversarial input |
| Judge from the candidate's family | Panickssery et al., Wataoka et al. | Judge from another family |
| Lower bound of kappa under 0.60 | McHugh: below 0.60 is inadequate agreement | Triage only, humans decide |
| After a model, prompt, or domain change | Zheng et al.: examples may not reproduce on later GPT-4 | Re-validate on the holdout |
| Humans disagree on the rubric | Human agreement sets the ceiling | Fix the rubric first |

Attacks are a real category of failure here. [Raina et al. (2024)](https://arxiv.org/abs/2402.14016) showed short adversarial phrases can inflate judge scores and transfer to unseen models, and found absolute scoring more exposed than comparison.

[Shi et al. (2024)](https://arxiv.org/abs/2403.17710) showed optimization-based prompt injection against judges, and reported that known-answer and perplexity detection were not enough.

The threshold is an editorial choice. [McHugh (2012)](https://www.biochemia-medica.com/en/journal/22/3/10.11613/BM.2012.031) treats a kappa below 0.60 as inadequate for health research, and no comparable standard exists for LLM judges.

Use the lower end of the interval, because a point estimate near the threshold is exactly when you cannot tell. Also set a ceiling on the false-pass rate, because that error is the one that ships.

## Start with one judge and one holdout

Start your first LLM as a judge check with one criterion that a person already reviews by hand. Have that person label 200 real outputs with failures oversampled, run the validation above, and only then let the judge run on new traffic. That one cycle tells you whether a judge belongs in your pipeline.

Apply this decision rule to the holdout result:

- If the lower bound of kappa clears 0.60 and the false-pass ceiling is acceptable, let the judge gate that criterion.
- If the point estimate clears 0.60 and the lower bound does not, use the judge to rank items for human review.
- If the point estimate is under 0.60, change the rubric or keep the check with people.

Run the position swap and the length padding test before either of the first two outcomes counts. For how this slots into a release process, see the [LLM evaluation guide](https://aifordevs.org/articles/llm-evaluation) and the [evals hub](https://aifordevs.org/topics/evals).

The [context engineering guide](https://aifordevs.org/articles/context-engineering) covers the layout changes you will most often be judging. [What is an AI agent](https://aifordevs.org/articles/what-is-an-ai-agent) covers where judge checks fit in an [agent loop](https://aifordevs.org/glossary#agent-loop), and [RAG vs fine-tuning](https://aifordevs.org/articles/rag-vs-fine-tuning) covers the retrieval systems whose faithfulness you may need to grade.

## Questions

### What is LLM as a judge?

LLM as a judge means using one language model to grade another model's output against a written criterion or by comparing two answers. It replaces or supplements human review for open-ended qualities such as whether an answer is complete or faithful to its sources. Treat the result as a measurement that must be calibrated against human labels.

### Is GPT-4 as accurate as a human judge?

Zheng et al. (2023) reported 85 percent agreement between GPT-4 and humans on MT-Bench with ties excluded, above the 81 percent between humans on the same subset. With ties and inconsistent votes counted, GPT-4 agreed 66 percent of the time. Those numbers describe 2023 models on one benchmark, so measure your own judge on your own task.

### What biases affect LLM judges?

Primary studies document position bias, where the slot an answer occupies changes the verdict; verbosity bias, where longer answers win more often; and self-preference, where a judge favors text from its own model family. Zheng et al. could not determine self-enhancement bias, while later work reports it for GPT-4. Swap answer order, control for length, and use a different family.

### How do I validate an LLM judge?

Label a sample of real outputs by hand, split it into a development set and a sealed holdout, and tune the rubric only on the development set. Then score the holdout once and report raw agreement, Cohen's kappa with a bootstrap interval, and the false-pass rate. Re-validate whenever the judge model, prompt, or task changes.

### What is a good Cohen's kappa for an LLM judge?

No standard exists for LLM judges. McHugh's 2012 review of the statistic treats any kappa below 0.60 as inadequate agreement for health research, and this article borrows that floor. Use the lower end of the confidence interval and check the false-pass rate, because a kappa near 0.6 can still let a third of bad outputs through.

### Can an LLM judge be attacked?

Yes. Raina et al. (2024) showed that short universal adversarial phrases can inflate a judge's scores and transfer to other models, with absolute scoring more exposed than comparison. Shi et al. (2024) showed optimization-based prompt injection against judges and reported that tested defenses were not enough. Do not rely on a judge alone for text an attacker can write.

## Sources

1. [Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (2023)](https://arxiv.org/abs/2306.05685)
2. [Wang et al., Large Language Models are not Fair Evaluators (2023)](https://arxiv.org/abs/2305.17926)
3. [Shi et al., Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge (2024)](https://arxiv.org/abs/2406.07791)
4. [Saito et al., Verbosity Bias in Preference Labeling by Large Language Models (2023)](https://arxiv.org/abs/2310.10076)
5. [Dubois et al., Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators (2024)](https://arxiv.org/abs/2404.04475)
6. [Panickssery et al., LLM Evaluators Recognize and Favor Their Own Generations (2024)](https://arxiv.org/abs/2404.13076)
7. [Wataoka et al., Self-Preference Bias in LLM-as-a-Judge (2024)](https://arxiv.org/abs/2410.21819)
8. [Raina et al., Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment (2024)](https://arxiv.org/abs/2402.14016)
9. [Shi et al., Optimization-based Prompt Injection Attack to LLM-as-a-Judge (2024)](https://arxiv.org/abs/2403.17710)
10. [Lee et al., How to Correctly Report LLM-as-a-Judge Evaluations (2025)](https://arxiv.org/abs/2511.21140)
11. [McHugh, Interrater reliability: the kappa statistic (2012)](https://www.biochemia-medica.com/en/journal/22/3/10.11613/BM.2012.031)
12. [scikit-learn docs: cohen_kappa_score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.cohen_kappa_score.html)
13. [SciPy docs: scipy.stats.bootstrap](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bootstrap.html)
14. [OpenAI docs: Evaluation best practices](https://developers.openai.com/api/docs/guides/evaluation-best-practices)
15. [Anthropic docs: Define success criteria and build evaluations](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests)

## Cite this deep dive

AI for Devs. "LLM as a judge: biases, validation, and when to distrust it." Published November 8, 2026. https://aifordevs.org/articles/llm-as-a-judge

- Page: https://aifordevs.org/articles/llm-as-a-judge
- Markdown: https://aifordevs.org/articles/llm-as-a-judge.md
- Reuse: Quote or summarize with a link to this page. Terms: https://aifordevs.org/reuse
- Publisher: AI for Devs
- Published: 2026-11-08