> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Basic Agent

> Run a basic agent against a local vLLM server.

## Code

```python vllm_basic.py theme={null}
from agno.agent import Agent
from agno.models.vllm import VLLM

agent = Agent(
    model=VLLM(id="Qwen/Qwen2.5-7B-Instruct", top_k=20, enable_thinking=False),
    markdown=True,
)

agent.print_response("Share a 2 sentence horror story")
```

> [!NOTE]
>
> Install vLLM on a [supported platform](https://docs.vllm.ai/en/latest/getting_started/installation/) with a compatible accelerator. Keep the model server running in a separate terminal while you run the agent.

## Usage

<Snippet file="create-venv-step.mdx" />

**1. Install dependencies**

```bash theme={null}
uv pip install -U agno openai vllm
```

**2. Start vLLM server**

In a separate terminal with the virtual environment active, run:

```bash theme={null}
vllm serve Qwen/Qwen2.5-7B-Instruct \
    --enable-auto-tool-choice \
    --tool-call-parser hermes \
    --dtype float16 \
    --max-model-len 8192 \
    --gpu-memory-utilization 0.9
```

**3. Set your API key**

```bash theme={null}
export VLLM_API_KEY=xxx
```

**4. Run Agent**

Save the code above as `vllm_basic.py`, then run:

```bash theme={null}
python vllm_basic.py
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.