> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM as judge

> Score model outputs against a rubric. The same machinery as labeling, applied to evaluation.

A judge is a classifier whose input is a `(prompt, response)` pair and whose output is a score. Use `int` with `ge` and `le` so validated scores stay on scale.

```python theme={null}
from agno.agent import Agent
from agno.models.google import Gemini
from pydantic import BaseModel, Field


class Score(BaseModel):
    overall: int = Field(
        ..., ge=1, le=5, description="Overall quality, 5 is excellent"
    )


agent = Agent(
    model=Gemini(id="gemini-3.5-flash"),
    instructions=(
        "Score the response on overall quality from 1 (unusable) to 5 "
        "(excellent). Use the full scale. Reserve 5 for genuinely "
        "excellent responses."
    ),
    output_schema=Score,
)


def build_input(prompt: str, response: str) -> str:
    return f"Prompt:\n{prompt}\n\nResponse:\n{response}"


prompt = "Explain why the sky is blue, in one sentence."
result = agent.run(build_input(prompt, "It just is.")).content
# Score(overall=1)
```

## Add a rationale

A free-text rationale makes the score auditable and surfaces rubric drift.

```python theme={null}
from pydantic import BaseModel, Field


class Score(BaseModel):
    overall: int = Field(..., ge=1, le=5, description="Overall quality")
    rationale: str = Field(..., description="Why this score, citing the response")
```

## Multi-dimension rubric

Use one bounded field per rubric dimension and a separate field for the overall assessment.

```python theme={null}
from pydantic import BaseModel, Field


class RubricScore(BaseModel):
    correctness: int = Field(..., ge=1, le=5, description="Factually correct")
    completeness: int = Field(..., ge=1, le=5, description="Covers what was asked")
    clarity: int = Field(..., ge=1, le=5, description="Easy to follow")
    concision: int = Field(..., ge=1, le=5, description="No padding")
    overall: int = Field(..., ge=1, le=5, description="Holistic quality")
```

## Picking the shape

| You need | Schema |
| - | - |
| One quality number | `int` with `ge=1, le=5` |
| Number plus justification | Add a `rationale` field |
| Per-criterion breakdown | One bounded `int` field per dimension |
| Pairwise comparison | [Preference data](https://agno-v2.mintlify.site/use-cases/data-labeling/preference-data.md) |

## Relationship to evals

This applies a classification schema to model outputs. When the judge is the deliverable, it lives here. When it scores a system under test, see [Evals](https://agno-v2.mintlify.site/evals/overview.md).

## Next steps

| Task | Guide |
| - | - |
| Rank two responses | [Preference data](https://agno-v2.mintlify.site/use-cases/data-labeling/preference-data.md) |
| Reduce single-model bias | [Quality pipeline](https://agno-v2.mintlify.site/use-cases/data-labeling/quality-pipeline.md) |

## Developer Resources

* [LLM-as-judge cookbook](https://github.com/agno-agi/agno/tree/main/cookbook/data_labeling/_17_llm_as_judge)
* [Evals](https://agno-v2.mintlify.site/evals/overview.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.