Skip to content
Docs menu

Limits and defaults

The built-in size caps, timeouts, queues and counts of the orchestrator, venom serve, Venom Cloud and the CLI, and where to change each one.

View .md

A limit you hit shows up as an error code (Error codes). This page lists the defaults and the setting that changes each one.

Orchestrator

Set in config/orchestrator.yaml. Keys under defaults: apply to every upstream unless the upstream sets its own.

Limit Default Setting
Request body size 32 MiB defaults.max_body_bytes (413 request_too_large)
Time for the request body to arrive 60 s fixed (408 request_timeout)
Upstream request timeout 120 s defaults.timeout, or timeout on an upstream
Gap between stream chunks before the stream is cut 5 min defaults.stream_idle_timeout
Client not reading a stream before it is dropped 60 s defaults.write_stall_timeout
Attempts per request across targets 3 defaults.max_attempts
Total time spent retrying 20 s defaults.retry_budget
Retry backoff 100 ms base, 2 s max defaults.backoff_base, defaults.backoff_max
Retry rate 20 % of requests, at least 1 per second defaults.retry_ratio, defaults.retry_min_per_second
Failures before a target's circuit opens 3 defaults.breaker_threshold
Circuit cooldown 1 min, then 5 min, then 30 min defaults.breaker_cooldowns (or one fixed defaults.breaker_cooldown)
Health check interval 15 s defaults.health_interval, or per upstream
Engine load poll interval 2 s defaults.load_interval
Engine treated as saturated KV cache 95 % full, or 4 requests waiting defaults.saturation_kv, defaults.saturation_waiting
Concurrent requests per upstream unlimited defaults.max_inflight, or per upstream
Request log rows kept 100,000 defaults.request_log_retention
Side-request response cache 5 min, 32 MiB total, 1 MiB per entry response_cache.ttl, response_cache.max_bytes, response_cache.max_entry_bytes
Idempotency-Key answers kept 5 min, 32 MiB dedup.idempotency_ttl, dedup.idempotency_max_bytes
Request dedup buffer 1 MiB per joined request, 256 KiB lag dedup.max_bytes, dedup.max_lag_bytes
Rate shaper queue 256 requests, 30 s wait rate_shaping.max_queue, rate_shaping.max_wait
Batch routing window 30 s, 1,000 lines per batch (at most 50,000) batch_routing.window, batch_routing.max_lines
Batch wait (x-venom-batch-wait) at most 10 min (at most 1 h) batch_routing.max_wait
Shadow traffic 1 replay at a time (at most 8), queue of 256, 1 MiB captured routes[].shadow.max_concurrency, routes[].shadow.queue, routes[].shadow.max_capture_bytes; routes[].shadow.daily_usd is required
Per-route rate limits none routes[].rate_limit.rpm, routes[].rate_limit.tpm (429 rate_limit_exceeded, 429 tpm_limit_exceeded)
Prewarm request 1 to 8 model ids, keep-alive 1 h (at most 12 h) fixed
X-Venom-Intent length 128 bytes fixed
Distinct intents tracked 100,000 fixed (503 intent_ledger_full)
Idempotency-Key length 255 bytes fixed

Per-key requests per minute, tokens per minute and budgets are set on each client key through the admin API.

venom serve

Set through venom serve flags or the server's options when you embed @venom/harness-server.

Limit Default
Open sessions 8 (429 too_many_sessions)
Open connections max(64, 4 × sessions)
Idle session timeout 30 min
Request body 1 MiB (413 body_too_large)
Prompt body, which may carry images 16 MiB, never below the body limit
Bearer token length at least 16 characters
Browser IDE editor UTF-8 text files up to 512 KiB

Venom Cloud

Limit Value
Request body through the relay 4 MiB (413 request_too_large); direct mode has no cloud-side cap
Idempotency-Key at most 255 characters, remembered for 24 h per API key
Log export 50,000 rows (x-venom-export-limit)
Relayed stream length a time limit per stream (stream_time_limit), after which the client resumes

Rate limits, budgets, spend caps and plan limits are set per organisation, key group and key in the cloud console.

CLI and harness

Limit Default Setting
Wait for a file lease another agent holds 30 s VENOM_LEASE_WAIT_MS
Auto-mode review time no limit beyond the request's VENOM_AUTO_TIMEOUT_MS, permissions.auto.timeout_ms
Chat width 100 columns VENOM_MAX_WIDTH, /width (40 to 1000)
Models prewarmed when a gear engages at most 8, the same set at most once per 5 min fixed
x-venom-task value 128 characters fixed