> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Integrate LiteLLM with Agno for a unified LLM experience.

[LiteLLM](https://docs.litellm.ai/docs/) provides a unified interface for various LLM providers, allowing you to use different models with the same code.

Agno integrates with LiteLLM in two ways:

1. **Direct SDK integration** - Using the LiteLLM Python SDK
2. **[Proxy Server integration](https://agno-v2.mintlify.site/models/providers/gateways/litellm-openai/overview.md)** - Using LiteLLM as an OpenAI-compatible proxy

## Prerequisites

For both integration methods, you'll need:

```shell theme={null}
# Install required packages
uv pip install agno litellm
```

Set up your API key:
Agno checks `LITELLM_API_KEY` first. If it's not set, it falls back to the provider-specific key for the model you're using, e.g. `OPENAI_API_KEY` for an OpenAI model.

```shell theme={null}
export LITELLM_API_KEY=your_api_key_here
```

## SDK Integration

The `LiteLLM` class provides direct integration with the LiteLLM Python SDK.

### Basic Usage

```python theme={null}
from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="gpt-5-mini",  # Model ID to use
        name="LiteLLM",  # Optional display name
    ),
    markdown=True,
)

# Get a response
agent.print_response("Share a 2 sentence horror story")
```

### Using Hugging Face Models

LiteLLM can also work with Hugging Face models:

```python theme={null}
from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="huggingface/mistralai/Mistral-7B-Instruct-v0.2",
        top_p=0.95,
    ),
    markdown=True,
)

agent.print_response("What's happening in France?")
```

### Configuration Options

The `LiteLLM` class accepts the following parameters:

| Parameter | Type | Description | Default |
| - | - | - | - |
| `id` | str | Model identifier (e.g., "gpt-5-mini" or "huggingface/mistralai/Mistral-7B-Instruct-v0.2") | "gpt-4o" |
| `name` | str | Display name for the model | "LiteLLM" |
| `provider` | str | Provider name | "LiteLLM" |
| `api_key` | Optional\[str] | API key (falls back to LITELLM\_API\_KEY environment variable) | None |
| `api_base` | Optional\[str] | Base URL for API requests | None |
| `max_tokens` | Optional\[int] | Maximum tokens in the response | None |
| `temperature` | float | Sampling temperature | 0.7 |
| `top_p` | float | Top-p sampling value | 1.0 |
| `request_params` | Optional\[Dict\[str, Any]] | Additional request parameters | None |

### Examples

> [!NOTE]
>
> View more examples [here](https://agno-v2.mintlify.site/models/providers/gateways/litellm/usage/basic-stream.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.