> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Spider Tools

> Search, scrape, and crawl websites into LLM-ready Markdown with SpiderTools.

```python spider_tools.py theme={null}
"""
Spider Tools
=============================

Demonstrates spider tools.
"""

from agno.agent import Agent
from agno.tools.spider import SpiderTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------


# Example 1: All functions available (default behavior)
agent_all = Agent(
    name="Spider Agent - All Functions",
    tools=[SpiderTools(optional_params={"proxy_enabled": True})],
    instructions=["You have access to all Spider web scraping capabilities."],
    markdown=True,
)

# Example 2: Include specific functions only
agent_specific = Agent(
    name="Spider Agent - Search Only",
    tools=[SpiderTools(enable_crawl=False, optional_params={"proxy_enabled": True})],
    instructions=["You can only search the web, no scraping or crawling."],
    markdown=True,
)


# Use the default agent for examples
agent = agent_all

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    agent.print_response(
        'Can you scrape the first search result from a search on "news in USA"?'
    )
```

## Run the Example

<Snippet file="create-venv-step.mdx" />

**1. Install dependencies**

```bash theme={null}
uv pip install -U agno openai spider-client
```

**2. Export your API keys**

```bash Mac/Linux theme={null}
export OPENAI_API_KEY="your_openai_api_key_here"
export SPIDER_API_KEY="your_spider_api_key_here"
```

```bash Windows theme={null}
$Env:OPENAI_API_KEY="your_openai_api_key_here"
$Env:SPIDER_API_KEY="your_spider_api_key_here"
```

**3. Run the example**

Save the code above as `spider_tools.py`, then run:

```bash theme={null}
python spider_tools.py
```

Full source: [cookbook/91\_tools/spider\_tools.py](https://github.com/agno-agi/agno/blob/v3.0.4/cookbook/91_tools/spider_tools.py)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.