> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# vLLM

> Parameter reference for VLLMEmbedder, which embeds locally with a vLLM engine or remotely via an OpenAI-compatible server.

The vLLM Embedder provides high-performance embedding inference with support for both local and remote deployment modes. It can load models directly for local inference or connect to a remote vLLM server via an OpenAI-compatible API.

## Usage

```python theme={null}
from agno.knowledge.embedder.vllm import VLLMEmbedder
from agno.knowledge.knowledge import Knowledge
from agno.vectordb.pgvector import PgVector

# Local mode
embedder = VLLMEmbedder(
    id="intfloat/e5-mistral-7b-instruct",
    dimensions=4096,
    enforce_eager=True,
    vllm_kwargs={
        "disable_sliding_window": True,
        "max_model_len": 4096,
    },
)

# Use with Knowledge
knowledge = Knowledge(
    vector_db=PgVector(
        db_url="postgresql+psycopg://ai:ai@localhost:5532/ai",
        table_name="vllm_embeddings",
        embedder=embedder,
    ),
)
```

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `id` | `str` | `"sentence-transformers/all-MiniLM-L6-v2"` | Model identifier (HuggingFace model name) |
| `dimensions` | `int` | `4096` | Embedding vector dimensions |
| `base_url` | `Optional[str]` | `None` | Remote vLLM server URL (enables remote mode) |
| `api_key` | `Optional[str]` | `getenv("VLLM_API_KEY")` | API key for remote server authentication |
| `enable_batch` | `bool` | `False` | Enable batch processing for multiple texts |
| `batch_size` | `int` | `100` | Number of texts to process per batch |
| `enforce_eager` | `bool` | `True` | Use eager execution mode (local mode) |
| `vllm_kwargs` | `Optional[Dict[str, Any]]` | `None` | Additional vLLM engine parameters (local mode) |
| `vllm_client` | `Optional[LLM]` | `None` | Pre-configured vLLM engine instance (local mode) |
| `request_params` | `Optional[Dict[str, Any]]` | `None` | Additional request parameters (remote mode) |
| `client_params` | `Optional[Dict[str, Any]]` | `None` | OpenAI client configuration (remote mode) |
| `remote_client` | `Optional[OpenAI]` | `None` | Pre-configured OpenAI-compatible client for the vLLM server (remote mode) |
| `async_remote_client` | `Optional[AsyncOpenAI]` | `None` | Pre-configured async OpenAI-compatible client for the vLLM server (remote mode) |

## Developer Resources

* [Local mode cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/07_knowledge/09_archive/embedders/vllm_embedder_local.py)
* [Remote mode cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/07_knowledge/09_archive/embedders/vllm_embedder_remote.py)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.