> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# ReliabilityEval

> ReliabilityEval parameters, tool call and argument checks, and ReliabilityResult fields.

`ReliabilityEval` checks the tool calls in a completed agent or team run against expected tool names and arguments.

```python theme={null}
from agno.eval import ReliabilityEval, ReliabilityResult
```

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `name` | `Optional[str]` | `None` | Evaluation name. |
| `eval_id` | `str` | Autogenerated UUID | Evaluation ID. |
| `agent_response` | `Optional[RunOutput]` | `None` | Agent run to check. Provide exactly one of `agent_response` or `team_response`. |
| `team_response` | `Optional[TeamRunOutput]` | `None` | Team run to check, including member responses. Provide exactly one of `agent_response` or `team_response`. |
| `expected_tool_calls` | `Optional[List[str]]` | `None` | Tool names that must appear in the run. Missing names fail the evaluation. |
| `allow_additional_tool_calls` | `bool` | `False` | Treat tool calls outside `expected_tool_calls` as allowed extras instead of failures. |
| `expected_tool_call_arguments` | `Optional[Dict[str, Union[Dict[str, Any], List[Dict[str, Any]]]]]` | `None` | Expected arguments per tool name, matched as a subset of the actual arguments. A list of dicts requires each spec to match at least one call to that tool. |
| `print_results` | `bool` | `False` | Print the results table after the run. |
| `show_spinner` | `bool` | `True` | Render the progress spinner. Set to `False` when the eval runs inside code that must not write to the console. |
| `file_path_to_save_results` | `Optional[str]` | `None` | File path where the result is saved. |
| `debug_mode` | `bool` | `False` | Enable debug logs. Defaults to `True` when the `AGNO_DEBUG` environment variable is `"true"`. |
| `db` | `Optional[Union[BaseDb, AsyncBaseDb]]` | `None` | Database where the eval run is stored. |
| `telemetry` | `bool` | `True` | Log minimal telemetry for the eval run. |
| `result` | `Optional[ReliabilityResult]` | `None` | Result of the evaluation. Set after a run completes. |

## Methods

| Method | Return type | Description |
| - | - | - |
| `run(*, print_results=False)` | `Optional[ReliabilityResult]` | Check the response's tool calls against the expectations. Raises with an async database; use `arun()` instead. |
| `arun(*, print_results=False)` | `Optional[ReliabilityResult]` | Async version of `run()`. |

## ReliabilityResult

The evaluation passes when there are no failed tool calls, no missing tool calls, and no failed argument checks.

| Field | Type | Description |
| - | - | - |
| `eval_status` | `str` | `"PASSED"` or `"FAILED"`. |
| `failed_tool_calls` | `List[str]` | Tool calls outside `expected_tool_calls` when additional calls are not allowed. |
| `passed_tool_calls` | `List[str]` | Tool calls that matched `expected_tool_calls`. |
| `additional_tool_calls` | `List[str]` | Tool calls outside `expected_tool_calls`, recorded when `allow_additional_tool_calls=True`. |
| `missing_tool_calls` | `List[str]` | Expected tool names that never fired. |
| `failed_argument_checks` | `List[str]` | Tool names whose arguments did not match `expected_tool_call_arguments`. |
| `passed_argument_checks` | `List[str]` | Tool names whose arguments matched `expected_tool_call_arguments`. |

`ReliabilityResult` also exposes `print_eval(console=None)` to render the results table, and `assert_passed()`, which asserts `eval_status == "PASSED"` for use in test suites.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.