> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Kubernetes Reference

> Commands, customization, environment variables, and troubleshooting for the Kubernetes template.

The scripts install the Helm release `agentos` into the `agentos` namespace. Override with `AGENTOS_RELEASE` and `AGENTOS_NAMESPACE`.

## Manage

| Task | Command |
| - | - |
| Roll to a new image tag | `IMAGE_TAG=v2 ./scripts/k8s/redeploy.sh` (build and push the tag first) |
| Restart pods in place | `./scripts/k8s/redeploy.sh` |
| Sync supported env variables | `./scripts/k8s/env-sync.sh` (updates nonempty values from a fixed allowlist that includes `RUNTIME_ENV`; defaults to `.env.production`, or pass `.env`) |
| Tail logs | `kubectl logs deploy/agentos -n agentos -f` |
| Port-forward the API | `kubectl port-forward svc/agentos 8000:8000 -n agentos` |
| Roll back a release | `helm rollback agentos -n agentos` |
| Tear down | `./scripts/k8s/down.sh` (add `--yes` to skip the confirmation) |

## Production auth

Token-Based Authorization is on by default. Production startup requires `JWT_VERIFICATION_KEY` or a readable JWKS file at the pod path in `JWT_JWKS_FILE`; otherwise the process exits.

Token-Based Auth gives you three things:

1. **No public access.** The server rejects requests without a valid token.
2. **Per-request identity.** Middleware validates the token and exposes its `user_id`, optional `session_id`, scopes, and claims to the request.
3. **Scope-based permissions.** Token scopes control access to AgentOS routes and resources.

The templates do not enable per-user data isolation. To scope non-admin session, memory, trace, and run access to the JWT subject, pass `authorization_config=AuthorizationConfig(user_isolation=True)` to `AgentOS`. See [User Isolation](https://agno-v2.mintlify.site/agent-os/security/authorization/user-isolation.md).

To opt out (not recommended), set `authorization=False` in `app/main.py`, then build and push a new image tag and roll to it with `IMAGE_TAG=<tag> ./scripts/k8s/redeploy.sh`. Use this only inside a private VPC behind another auth layer. Without it, anyone who guesses your AgentOS URL can access your platform.

## Customize

> **Add an agent**
>
> Ask your coding agent to run `/create-agent`, or do it by hand. Create `agents/my_agent.py`:
>
> ```python theme={null}
> from agno.agent import Agent
>
> from app.settings import default_model
> from db import get_postgres_db
>
> INSTRUCTIONS = """\
> What the agent does, which tools it uses, the rules to follow when answering.
> """
>
> my_agent = Agent(
>     id="my-agent",
>     name="My Agent",
>     model=default_model(),
>     db=get_postgres_db(),
>     instructions=INSTRUCTIONS,
>     enable_agentic_memory=True,
>     add_datetime_to_context=True,
>     add_history_to_context=True,
>     num_history_runs=5,
> )
> ```
>
> Register it in `app/main.py`:
>
> ```python theme={null}
> from agents.my_agent import my_agent
>
> agent_os = AgentOS(
>     ...
>     agents=[agent_builder, platform_manager, web_search, my_agent],
> )
> ```
>
> Add its UI metadata beneath the existing `manifest:` key in `app/config.yaml`:
>
> ```yaml theme={null}
>   my-agent:
>     description: "What the agent does."
>     quick_prompts:
>       - "First example prompt"
>       - "Second example prompt"
>       - "Third example prompt"
> ```
>
> Local containers hot-reload on save. For production, build and push a new image tag, then run `IMAGE_TAG=<tag> ./scripts/k8s/redeploy.sh`. If the release still runs the official image, point it at your registry first: `IMAGE_REPOSITORY=<registry>/agentos IMAGE_TAG=<tag> ./scripts/k8s/up.sh`.

> **Change the model**
>
> `app/settings.py` defines `default_model()`, used by every agent. Change it in one place:
>
> ```python theme={null}
> from agno.models.anthropic import Claude
>
> def default_model():
>     return Claude(id="claude-sonnet-5")
> ```
>
> Add `anthropic` to `pyproject.toml`, set the provider key in your env, and regenerate pins:
>
> ```bash theme={null}
> ./scripts/generate_requirements.sh
> ```
>
> Rebuild locally with `docker compose up -d --build`. For production, build and push a new tag, then roll to it:
>
> ```bash theme={null}
> docker build -t <registry>/agentos:v2 . && docker push <registry>/agentos:v2
> IMAGE_TAG=v2 ./scripts/k8s/redeploy.sh
> ```
>
> `env-sync.sh` uses a fixed allowlist that includes `RUNTIME_ENV`, `AGENTOS_URL`, the `JWT_JWKS_FILE` path, the template's supported secrets, and `DB_PASS`. It does not deliver the referenced JWKS file or sync a new provider key such as `ANTHROPIC_API_KEY`. Provide the JWKS file through a custom image or chart volume. Deliver a new provider key via `extraEnv` and `helm upgrade`.

> **Add tools**
>
> Agno ships 100+ toolkits. See [Toolkits](https://agno-v2.mintlify.site/tools/toolkits/overview.md).
>
> ```python theme={null}
> from agno.tools.slack import SlackTools
>
> my_agent = Agent(
>     ...
>     tools=[SlackTools()],
> )
> ```

> **Add dependencies**
>
> 1. Edit `pyproject.toml`.
> 2. Regenerate pins: `./scripts/generate_requirements.sh` (add `upgrade` to refresh every pin).
> 3. Rebuild locally with `docker compose up -d --build`, or build and push a new tag and roll to it with `IMAGE_TAG=<tag> ./scripts/k8s/redeploy.sh`.

> **Enable Slack**
>
> Set both variables in your env file:
>
> ```bash theme={null}
> SLACK_BOT_TOKEN=xoxb-...
> SLACK_SIGNING_SECRET=...
> ```
>
> Sync with `./scripts/k8s/env-sync.sh`. The interface activates automatically and routes messages to Agent Builder; change the `agent=` argument in `app/main.py` to point at another agent. See [Slack setup](https://agno-v2.mintlify.site/agent-os/interfaces/slack/setup.md).

> **Toggle scheduled workflows**
>
> The deployment check runs daily by default (`ENABLE_DEPLOY_CHECK=True`); it is deterministic and free. The `run-evals` schedule is always registered but starts disabled because it uses model calls. Enable it from the AgentOS UI. Both workflows remain runnable on demand.
>
> In the cluster these are chart values. Set `ENABLE_DEPLOY_CHECK` and `EVALS_*` via `extraEnv` and `helm upgrade`; `env-sync.sh` does not sync them. Enable the registered `run-evals` schedule from the AgentOS UI.

## Format, validate, and run evals

The format, validate, and eval scripts run on the host and need a venv. Set it up once:

```bash theme={null}
./scripts/venv_setup.sh
source .venv/bin/activate
```

| Task | Command |
| - | - |
| Format | `./scripts/format.sh` |
| Lint and type-check | `./scripts/validate.sh` |
| Run smoke evals | `python -m evals --tag smoke` |

`./scripts/mcp_check.sh` runs inside the container, so it needs no venv.

## Environment variables

| Variable | Required | Default | Description |
| - | - | - | - |
| `OPENAI_API_KEY` | Yes | - | Models and embeddings. |
| `RUNTIME_ENV` | No | `prd` | `dev` disables JWT. Compose sets it for local. Never put it in an env file that syncs to a real cluster, or production deploys unauthenticated. |
| `JWT_VERIFICATION_KEY` | Production | - | Public key from os.agno.com. Quote the value so the multi-line PEM parses as one variable. |
| `JWT_JWKS_FILE` | Production | - | Path inside the pod to a JWKS file. `up.sh` and `env-sync.sh` set only the `jwtJwksFile` path. The current chart does not mount the file. |
| `MCP_CONNECT_SECRET` | No | - | OAuth consent secret (16+ chars) for connecting claude.ai and ChatGPT to `/mcp`. `up.sh` generates it into `.env.production` when the deploy has a public URL (`INGRESS_HOST` or `AGENTOS_URL`); set it by hand otherwise. |
| `AGENTOS_MCP_SIGNING_KEY` | No | generated | Optional high-entropy signing-key material (32+ chars) for OAuth tokens. Unset, a strong key is generated and persisted in the database. Rotating it invalidates outstanding tokens. |
| `AGENTOS_URL` | No | `http://agentos:8000` | Scheduler base URL. The chart resolves an explicit value first, then the ingress URL, then the release service URL. Set it only for a custom domain or tunnel. When `MCP_CONNECT_SECRET` is set, OAuth metadata uses this URL as its public origin. |
| `ENABLE_DEPLOY_CHECK` | No | `True` | Daily deployment-check cron. |
| `EVALS_TAG` | No | `smoke` | Eval tag the run-evals workflow runs. |
| `EVALS_CASE_TIMEOUT_SECONDS` | No | `90` | Per-case timeout for run-evals runs. |
| `EVALS_SUITE_TIMEOUT_SECONDS` | No | `900` | Whole-suite timeout for run-evals runs. |
| `PARALLEL_API_KEY` | No | - | WebSearch uses the Parallel SDK when set, keyless MCP otherwise. |
| `SLACK_BOT_TOKEN` | No | - | Set with the signing secret to enable Slack. |
| `SLACK_SIGNING_SECRET` | No | - | Set with the bot token to enable Slack. |
| `DB_HOST` / `DB_PORT` / `DB_USER` / `DB_PASS` / `DB_DATABASE` | No | matches compose | Postgres connection. `up.sh` generates `DB_PASS` once and saves it to your env file. |
| `DB_DRIVER` | No | `postgresql+psycopg` | SQLAlchemy driver. |
| `AGNO_DEBUG` | No | `False` | Verbose Agno logs. Compose sets it for dev. |
| `WAIT_FOR_DB` | No | `True` in Helm | If `True`, the entrypoint blocks on the database before starting. The Helm chart and Compose set it to `True`. |
| `AGENTOS_NAMESPACE` | No | `agentos` | Namespace the k8s scripts target. |
| `AGENTOS_RELEASE` | No | `agentos` | Helm release name the k8s scripts target. |
| `IMAGE_REPOSITORY` | No | `agnohq/agentos` | Image the chart deploys. Read by `up.sh`. |
| `IMAGE_TAG` | No | `latest` | Image tag. `up.sh` installs it; `redeploy.sh` rolls the release to it. |
| `IMAGE_PULL_POLICY` | No | `IfNotPresent` | Set `Never` for images loaded into kind. Read by `up.sh`. |
| `INGRESS_HOST` | No | - | Publishes the API behind your ingress controller at this host. Read by `up.sh`. |
| `INGRESS_CLASS` | No | - | Ingress class name, for example `nginx`. Read by `up.sh`. |

## Troubleshooting

> **kubectl or helm: command not found**
>
> Install [kubectl](https://kubernetes.io/docs/tasks/tools/) and [Helm](https://helm.sh/docs/intro/install/) 3+. The scripts check for both before doing anything.

> **up.sh exits: no context or cluster not reachable**
>
> `up.sh` deploys into your current kubectl context and verifies it can reach the cluster first. Point kubectl at the target cluster and confirm `kubectl get namespace` works, then rerun.

> **up.sh pauses asking for a JWT key**
>
> Expected. At [os.agno.com](https://os.agno.com), choose **Connect OS** → **Live**, enter your AgentOS URL, name it **Live AgentOS**, turn on **Token-Based Authorization (JWT)** on the connection panel, and connect. The UI generates the public key. If the OS is already connected, enable the setting under **Settings** → **OS & Security**. Paste the full PEM into the script prompt. To add a PEM later, set `JWT_VERIFICATION_KEY` and run `./scripts/k8s/env-sync.sh`. To use JWKS, first provide the file through a custom image or chart volume, then set `JWT_JWKS_FILE` to its pod path and sync.

> **App fails to start in production**
>
> JWT auth is on whenever `RUNTIME_ENV` is not `dev`. Set `JWT_VERIFICATION_KEY` and sync. `JWT_JWKS_FILE` works only when a custom image or chart mount already provides a readable file at that pod path. To opt out inside a private VPC behind another auth layer, set `authorization=False` in `app/main.py` and roll out your own image build.

> **Pods stuck in ImagePullBackOff**
>
> The cluster can't pull the image. Confirm the tag was pushed and the cluster has access to your registry; for private registries, set `imagePullSecrets` in `charts/agentos/values.yaml`. On kind, `kind load docker-image` the tag and deploy with `IMAGE_PULL_POLICY=Never`.

> **Database rejects the app after a password change**
>
> The Postgres volume reads its password only on first initialization, so a lost or regenerated `DB_PASS` locks the app out of an existing volume. Restore the `DB_PASS` that `up.sh` saved to your env file and sync, fix the database in place with `ALTER USER`, or delete the PVC to reinitialize. Deleting the PVC deletes all data.

> **Scheduled jobs never fire**
>
> `AGENTOS_URL` resolves automatically: explicit value, then ingress URL, then in-cluster service DNS. If you set it by hand, make sure the pod can reach that URL, then run `./scripts/k8s/env-sync.sh`.

> **claude.ai or ChatGPT can't connect to /mcp**
>
> `up.sh` generates `MCP_CONNECT_SECRET` only when the deploy has a public URL (`INGRESS_HOST` or an explicit `AGENTOS_URL`). Deployed without one? Set `MCP_CONNECT_SECRET` and a public `AGENTOS_URL` in `.env.production` and run `./scripts/k8s/env-sync.sh`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.