Skip to content
Docs menu

Reasoning-effort auto-tuning

Let the orchestrator pick a reasoning effort per request with reasoning_effort auto in a route profile, and what it will never override.

View .md

A fixed thinking level is too high for a rename and too low for a hard bug. With reasoning_effort: auto, the orchestrator picks the effort for each request from the request itself. It is off unless a route profile, or a side-request kind, asks for it.

profiles:
  tuned:
    defaults: {reasoning_effort: auto}                 # low..high
  deep:
    defaults: {reasoning_effort: auto, effort_min: medium, effort_max: xhigh}

The shipped config has tuned (low to high) and eval-auto (the eval profile's flex tier plus auto). Pick one with a gear's profile: or the x-venom-profile header.

What it picks

It reads only the last turns, with no model call and no state.

The request's last turn Effort
A tool result that looks like a failure, twice in a row xhigh
A tool result that looks like a failure high
Any other tool result (a follow-up) low
A user turn about planning, design, debugging, a migration, a refactor or security high
A short user turn about a typo, a rename, formatting or a comment low
A very long user turn high
Anything else medium

The pick is clamped to effort_min..effort_max (default low..high), then moved to a level the target model accepts, using the same per-provider tables as the effort adapter.

What it never does

  • Override you. A request that names reasoning_effort, reasoning, thinking, enable_thinking or thinking_budget is left alone. A gear's thinking level other than off is sent as an effort, so it wins too.
  • Touch a model that does not take it. Anthropic, local kinds (Ollama, vLLM, llama.cpp, SGLang), unknown models, and OpenAI chat requests with tools get nothing. OpenAI Responses requests with tools do.
  • Change the prompt. Only the effort field changes, so prompt caches are unaffected.

Recording

Every request row records the effort it went out with (reasoning_effort) and effort_auto (true when the orchestrator chose it), in the admin request log (GET /admin/v1/requests) next to tokens and cost.

Evaluating it

Run the paired eval compare rather than judging by eye: arm A at your current fixed effort through the eval profile, arm B with eval-auto, both on the same model. Dry-run first to see the cost, keep a focused --filter and a --max-usd cap, then compare pass@1, pass^k, and cost and time per trial.