> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LLMs.txt Reader

> Fetch an llms.txt index and convert its linked documentation pages into documents.

`LLMsTxtReader` creates a document for an [llms.txt](https://llmstxt.org) overview, then fetches up to `max_urls` linked pages.

```python llms_txt_reader.py theme={null}
from agno.agent import Agent
from agno.knowledge.knowledge import Knowledge
from agno.knowledge.reader.llms_txt_reader import LLMsTxtReader
from agno.models.openai import OpenAIResponses
from agno.vectordb.pgvector import PgVector

db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"

knowledge = Knowledge(
    name="LLMs.txt Docs",
    vector_db=PgVector(table_name="llms_txt_docs", db_url=db_url),
)

agent = Agent(
    model=OpenAIResponses(id="gpt-5.2"),
    knowledge=knowledge,
    search_knowledge=True,
)

if __name__ == "__main__":
    knowledge.insert(
        url="https://docs.agno.com/llms.txt",
        reader=LLMsTxtReader(
            max_urls=10,
            allowed_hosts=["docs.agno.com"],
        ),
    )
    agent.print_response("What is Agno?", markdown=True)
```

## Run the Agent

<Snippet file="create-venv-step.mdx" />

**1. Install dependencies**

```bash theme={null}
uv pip install -U agno beautifulsoup4 openai pgvector psycopg sqlalchemy
```

**2. Export the API key**

```bash theme={null}
export OPENAI_API_KEY=your_openai_api_key_here
```

<Snippet file="run-pgvector-step.mdx" />

**3. Run the agent**

```bash theme={null}
python llms_txt_reader.py
```

## Reader Parameters

<Snippet file="llms-txt-reader-reference.mdx" />

`allowed_hosts` applies to the index, linked pages, and redirects. `LLMsTxtReader.async_read()` fetches the linked pages concurrently.

## Next Steps

| Task | Guide |
| - | - |
| Restrict outbound requests | [Restricting URL Fetches](https://agno-v2.mintlify.site/knowledge/concepts/readers/overview.md#restricting-url-fetches) |
| Crawl pages without an llms.txt index | [Website Reader](https://agno-v2.mintlify.site/knowledge/concepts/readers/website-reader.md) |
| Configure content chunking | [Chunking Overview](https://agno-v2.mintlify.site/knowledge/concepts/chunking/overview.md) |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.