> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Trafilatura

> TrafilaturaTools wraps the trafilatura library to expose extract_text, extract_metadata_only, html_to_text, extract_batch, and crawl_website functions.

**TrafilaturaTools** enable an Agent to extract text and metadata from web pages using the Trafilatura library.

## Prerequisites

The following example requires the `trafilatura` library.

```shell theme={null}
uv pip install -U trafilatura openai
```

## Example

The following agent can extract and analyze web content:

```python theme={null}
from agno.agent import Agent
from agno.tools.trafilatura import TrafilaturaTools

agent = Agent(
    instructions=[
        "You are a web content extraction specialist",
        "Extract clean text and structured data from web pages",
        "Provide detailed analysis of web content and metadata",
        "Help with content research and web data collection",
    ],
    tools=[TrafilaturaTools()],
)

agent.print_response("Extract the main content from https://example.com/article", stream=True)
```

## Toolkit Params

| Parameter | Type | Default | Description |
| - | - | - | - |
| `output_format` | `str` | `"txt"` | Default output format (txt, json, xml, markdown, csv, html, xmltei). |
| `include_comments` | `bool` | `True` | Whether to extract comments along with main text. |
| `include_tables` | `bool` | `True` | Whether to include table content. |
| `include_images` | `bool` | `False` | Whether to include image information (experimental). |
| `include_formatting` | `bool` | `False` | Whether to preserve text formatting. |
| `include_links` | `bool` | `False` | Whether to preserve links (experimental). |
| `with_metadata` | `bool` | `False` | Whether to include metadata in extractions. |
| `favor_precision` | `bool` | `False` | Whether to prefer precision over recall. |
| `favor_recall` | `bool` | `False` | Whether to prefer recall over precision. |
| `target_language` | `Optional[str]` | `None` | Target language filter (ISO 639-1 format). |
| `deduplicate` | `bool` | `False` | Whether to remove duplicate segments. |
| `max_tree_size` | `Optional[int]` | `None` | Maximum tree size for processing. |
| `max_crawl_urls` | `int` | `10` | Maximum number of URLs to crawl per website. |
| `max_known_urls` | `int` | `100000` | Maximum number of known URLs during crawling. |
| `enable_extract_text` | `bool` | `True` | Whether to extract text content. |
| `enable_extract_metadata_only` | `bool` | `True` | Whether to extract metadata information. |
| `enable_html_to_text` | `bool` | `True` | Whether to convert HTML content to clean text. |
| `enable_extract_batch` | `bool` | `True` | Whether to extract content from multiple URLs in batch. |
| `enable_crawl_website` | `bool` | `True` | Whether to crawl a website and extract content from multiple pages. |
| `all` | `bool` | `False` | Enable all tools. |

## Toolkit Functions

| Function | Description |
| - | - |
| `extract_text` | Extract main text content from a URL. |
| `extract_metadata_only` | Extract metadata information from web pages. |
| `html_to_text` | Convert HTML content to clean text. |
| `crawl_website` | Crawl a website and extract content from multiple pages. |
| `extract_batch` | Extract content from multiple URLs in batch. |

## Developer Resources

* [Tools Source](https://github.com/agno-agi/agno/blob/main/libs/agno/agno/tools/trafilatura.py)
* [Trafilatura Documentation](https://trafilatura.readthedocs.io/)
* [Web Scraping Best Practices](https://trafilatura.readthedocs.io/en/latest/corefunctions.html)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.