---
title: Configuration reference
description: Every top-level section of orchestrator.yaml with its keys and defaults, plus how the file is loaded, reloaded, versioned and rolled back.
section: orchestrator
order: 40
audience: [operator, agent]
sources: [config/orchestrator.yaml, services/go-orchestrator/internal/config, config/prices.yaml, docs/architecture.md, services/go-orchestrator/AGENTS.md]
features: [orchestrator.config-listen, orchestrator.config-defaults, orchestrator.config-response-cache, orchestrator.config-dedup, orchestrator.config-rate-shaping, orchestrator.config-batch-routing, orchestrator.config-pricing, orchestrator.config-models, orchestrator.config-guardrails, orchestrator.admin-config, orchestrator.admin-reload]
---

The orchestrator reads one YAML file, `$VENOM_CONFIG` (default `config/orchestrator.yaml`). The shipped file is commented line by line and is the best starting point; this page lists every key.

## Loading and reloading {#loading}

- `${VAR}` and `${VAR:-default}` are expanded from the environment before parsing. An unset or empty variable with no default becomes `""`; a bare `$` is left alone.
- Parsing is strict: an unknown key, kind or strategy, a duplicate name, or a route target on an unknown upstream stops the load with the field named. Availability is lenient: an upstream whose key variable is empty is kept but disabled.
- Legal but likely-wrong settings (a `ttft_timeout` not shorter than `defaults.retry_budget`, a profile target outside its own `allow` list) are logged as warnings at every load.
- Reload without a restart: `POST /admin/v1/reload` or `SIGHUP`. Both also re-read the price table. A reload that fails keeps the running config.

```bash
curl -s -X POST http://127.0.0.1:8787/admin/v1/reload -H "Authorization: Bearer $VENOM_ADMIN_TOKEN"
```

## Top-level sections {#sections}

| Key | Page |
|---|---|
| `listen` | below |
| `upstreams` | [Upstreams and kinds](/docs/orchestrator/upstreams.md#keys) |
| `routes` | [Routes and strategies](/docs/orchestrator/routing.md#keys) |
| `profiles`, `side_requests` | [Profiles, side requests and gears](/docs/orchestrator/profiles-and-gears.md) |
| `gears`, `gears_locked`, `gears_source` | [Profiles, side requests and gears](/docs/orchestrator/profiles-and-gears.md#gears) |
| `defaults`, `response_cache`, `dedup`, `rate_shaping`, `batch_routing`, `pricing`, `models`, `guardrails` | below |
| `search` | [Web search and MCP](/docs/orchestrator/search-and-mcp.md#config) |

## listen {#listen}

`listen: ${VENOM_LISTEN:-127.0.0.1:8787}` in the shipped file; `127.0.0.1:8787` when absent. Loopback by default: with no client key the gateway serves anyone who can reach it, so it refuses to start on another address without a key unless `VENOM_ALLOW_OPEN=1`. The Docker image sets `VENOM_LISTEN=0.0.0.0:8787` inside the container and compose publishes the port on the host's loopback only.

## defaults {#defaults}

| Key | Default | What it does |
|---|---|---|
| `defaults.timeout` | `120s` | Per-attempt time to first byte. Once a byte arrives the request runs until it ends or the client leaves. |
| `defaults.max_inflight` | `0` (unlimited) | Concurrent requests per upstream, unless the upstream sets its own. |
| `defaults.health_interval` | `15s` | Health-check period per upstream. |
| `defaults.max_attempts` | `3` | Total attempts per request. Every target still gets one try when there are more targets; extra attempts retry the same targets with backoff. |
| `defaults.retry_budget` | `20s` | No new attempt starts after this much time. |
| `defaults.backoff_base`, `defaults.backoff_max` | `100ms`, `2s` | Full-jitter exponential backoff. `backoff_max` is also the longest upstream `Retry-After` waited on; a longer one goes to the client. |
| `defaults.retry_ratio`, `defaults.retry_min_per_second` | `0.2`, `1` | Per-upstream retry budget: retries are allowed while retries ≤ ratio × first attempts + min_per_second × 10 s over the last 10 s. Both 0 = no same-upstream retries. |
| `defaults.breaker_threshold` | `3` | Consecutive failures that open an upstream's circuit breaker. |
| `defaults.breaker_cooldowns` | `[1m, 5m, 30m]` | Growing cooldown per trip (the last repeats) before a half-open probe. |
| `defaults.breaker_cooldown` | none | One fixed cooldown instead of the ladder. |
| `defaults.max_body_bytes` | `33554432` (32 MiB) | Request body cap (`413 request_too_large`). Also caps audio uploads. |
| `defaults.write_stall_timeout` | `60s` | A client that takes no bytes for this long during one write is dropped and its upstream request cancelled. |
| `defaults.stream_idle_timeout` | `5m` | An upstream that sends nothing for this long mid-response is cut off; the stream ends with its error event. Raise it for models that think for minutes without a byte. |
| `defaults.request_log_retention` | `100000` | Request-log rows kept in SQLite (the spend ledger is never pruned). |
| `defaults.load_interval` | `2s` | How often `vllm`, `sglang` and `llamacpp` engines' `/metrics` are read for queue and KV-cache use. |
| `defaults.saturation_kv`, `defaults.saturation_waiting` | `0.95`, `4` | When an engine counts as saturated (KV cache that full, or that many requests queued); a saturated engine goes behind the others. |

## response_cache {#response-cache}

Exact-match reuse of deterministic side requests (a side kind at temperature 0, or any kind listed in `kinds`) from the same client key; never a main-session turn. A hit is billed $0 and answered with `x-venom-response-cache: hit`, or `shared` when an identical request was in flight.

| Key | Default | What it does |
|---|---|---|
| `response_cache.ttl` | `5m` | How long a response is reused. |
| `response_cache.max_bytes` | `33554432` (32 MiB) | Memory bound, least recently used evicted; negative turns the cache off. |
| `response_cache.max_entry_bytes` | `1048576` (1 MiB) | Largest response stored. |
| `response_cache.kinds` | `[]` (shipped: `[auto-classifier, decide]`) | Side kinds cached at any temperature. |

## dedup {#dedup}

Identical concurrent requests from one key (temperature 0 set explicitly, `/v1/embeddings`, `/v1/rerank`, or any request with the same `Idempotency-Key`) share one upstream call, fanned out as it streams. A request opts out with `x-venom-dedup: off`. Duplicates are billed once and answered with `x-venom-dedup: joined`; a completed `Idempotency-Key` response is replayed with `x-venom-dedup: replayed` and `Idempotent-Replayed: true`. The same key with another body is `422 idempotency_key_reused`; a key over 255 bytes is `400 invalid_idempotency_key`.

| Key | Default | What it does |
|---|---|---|
| `dedup.max_bytes` | `1048576` (1 MiB) | A duplicate may join while the response is shorter; a longer `Idempotency-Key` response is not kept. |
| `dedup.max_lag_bytes` | `262144` (256 KiB) | A reader that falls this far behind is cut loose, so a slow client never holds the others. |
| `dedup.idempotency_ttl` | `5m` | How long a completed `Idempotency-Key` response is replayed. |
| `dedup.idempotency_max_bytes` | `33554432` (32 MiB) | Bound on kept responses; `-1` keeps none (collapsed only while in flight). |

## rate_shaping {#rate-shaping}

Each upstream learns its provider's rate limits from `x-ratelimit-*` and `anthropic-ratelimit-*` headers and from `429`s. A request that would exceed a learned limit goes to a later target with room, else waits in that upstream's queue in fair order across client keys. A request that waited carries `x-venom-queued-ms`. Only a full queue or a wait past the bound answers `429 upstream_queue_full` or `upstream_wait_exceeded`, with `Retry-After`. Upstreams that never send rate-limit headers (local engines) are not shaped.

| Key | Default | What it does |
|---|---|---|
| `rate_shaping.max_queue` | `256` | Requests waiting per upstream; negative = none wait. |
| `rate_shaping.max_wait` | `30s` | Longest wait; negative = none wait (limits then only steer requests). |
| `rate_shaping.weights` | each key 1 | Fair-queuing weight by client key name (at most 1000). |

## batch_routing {#batch-routing}

Shapes batch requests (`x-venom-urgency: batch`); see [Batches, files and service tiers](/docs/orchestrator/batches-and-files.md#batch-requests).

| Key | Default | What it does |
|---|---|---|
| `batch_routing.window` | `30s` (at most 24h) | How long requests for one model collect before their provider batch is submitted. |
| `batch_routing.max_lines` | `1000` (at most 50,000) | Submit at once at this many requests. |
| `batch_routing.max_wait` | `10m` (at most 1h) | Cap on `x-venom-batch-wait` and `GET /v1/batch_requests/{id}?wait=`. |
| `batch_routing.callback_hosts` | `[]` (no callbacks) | Hosts (`host` or `host:port`) `x-venom-callback-url` may name. The orchestrator calls no client-named URL unless you list its host. |

## pricing {#pricing}

Costs are list prices from the offline price table (`prices.yaml` next to the config, or `$VENOM_PRICES`; never fetched at runtime): USD per 1M `input`, `cached_input` and `output` tokens, with optional `cache_write`, `flex` and `priority` rates. Local upstreams cost $0. A cloud model nothing prices costs $0 and is logged once; `GET /admin/v1/pricing` lists every model's effective price and source.

```yaml
pricing:
  source: [custom, builtin]
  models:
    azure/my-gpt: {base_model: gpt-5.4}                  # priced as gpt-5.4
    my-tuned:     {input: 3, cached_input: 0.3, output: 12}
    tts-hosted/narrator: {per_1m_chars: 15}              # text to speech, per 1M characters
```

| Key | Default | What it does |
|---|---|---|
| `pricing.source` | `[custom, builtin]` | Lookup order over your prices (`custom`) and the table (`builtin`); a source left out is not consulted. |
| `pricing.models` | none | Rules keyed by `upstream/model` (that upstream only) or a bare model id (any upstream). |
| `pricing.models.*.base_model` | none | The model whose price applies. A served model with a base model is answered with `x-venom-base-model`. |
| `pricing.models.*.input`, `cached_input`, `output` | none | Your price in USD per 1M tokens; `input` and `output` together, `cached_input` defaults to `input`. Wins over `base_model`. |
| `pricing.models.*.search_unit` | none | USD per search unit, for a reranker billed per search. |
| `pricing.models.*.per_1m_chars` | none | USD per 1M input characters, for text-to-speech. |
| `pricing.models.*.per_second` | none | USD per second of audio, for speech-to-text. |

The same pricing keys are accepted in an upstream's `model_info` and on a route target; one model priced two different ways is a config error.

## models {#models}

Per-model settings a mesh daemon advertises; today only provenance for the public mesh.

| Key | What it does |
|---|---|
| `models[].id` | A model id the daemon serves (a route alias or `upstream/model`). Unique. |
| `models[].upstream` | The `vllm` or `sglang` upstream that runs it; public jobs go to it alone. Required with `provenance`. |
| `models[].provenance.registry_id` | The public mesh registry entry it runs. |
| `models[].provenance.sampler` | Must be `venom-sampler/1`. |

## guardrails {#guardrails}

`guardrails:` holds the layered guardrails checked on every `/v1` inference request: the org's rules and catalog picks, `teams:` keyed by a client key's team label and `members:` keyed by a key's name. A block is `403 guardrail_blocked` with a code such as `GR-ORG-CRED-101`; a redaction, warning or keep-local match passes with an `x-venom-guardrail` header per rule. `GET /v1/guardrails` shows a key what applies to it, and `GET /g/{code}` (no key) is a code's page. Absent, nothing is checked and nothing costs. The rule format, codes and layers are in the [Security section](/docs/security.md).

## Config history and rollback {#history}

Every config the process applies (start, reload, `SIGHUP`, a local-model selection, a rollback) is stored in SQLite as the file's bytes before `${VAR}` expansion, with its SHA-256; the newest 50 are kept.

```bash
curl -s http://127.0.0.1:8787/admin/v1/config/versions -H "Authorization: Bearer $VENOM_ADMIN_TOKEN"
curl -s "http://127.0.0.1:8787/admin/v1/config/diff?from=12&to=file" -H "Authorization: Bearer $VENOM_ADMIN_TOKEN"
curl -s -X POST http://127.0.0.1:8787/admin/v1/config/versions/12/rollback \
  -H "Authorization: Bearer $VENOM_ADMIN_TOKEN" -H "Content-Type: application/json" -d '{}'
```

- `GET /admin/v1/config/versions` lists versions newest first, marks the `current` one and says whether the file on disk differs (`file.changed`).
- `GET /admin/v1/config/diff?from=<id|file>&to=<id|file>` diffs two versions (default: the newest and the one before) by upstream, route, profile and other key. Secrets are redacted: `${VAR}` stays as written, literal header values and secret-named fields become a short hash.
- `POST /admin/v1/config/versions/{id}/rollback` writes that version to the file and reloads it. `409 config_changed_on_disk` when the file isn't what was last applied (send `overwrite_sha256` equal to the reported `disk_sha256` to replace it anyway), `422 invalid_version` when it no longer loads, `500 config_not_writable` for a read-only mount, `422 reload_failed` restores the previous file.

## Related

- [Environment variables](/docs/orchestrator/environment.md), [Admin API](/docs/orchestrator/admin-api.md), [Operations](/docs/orchestrator/operations.md).
