Skip to content
Docs menu

Run venom fully offline with Ollama

Run the venom coding agent on local models with no network, from Ollama alone up to the orchestrator and an air-gapped install, and check that nothing left the machine.

View .md

venom makes no calls of its own: no telemetry, no update checks. With a local model server, the only traffic is between venom and that server. This recipe goes from the quickest setup to a machine that has never been online.

Setup What you run Pick it when
Ollama, direct Ollama and the CLI trying it out; one laptop
Orchestrator in front of Ollama Ollama, the orchestrator, the CLI you want fallback between local models, a request log, budgets, or several users on one GPU box
Air-gapped the same, installed from a bundle you carry in the machine has no internet at all

Pick a model that calls tools

The agent works by calling tools, so the model must return structured tool calls. The qwen3 family does, and it is what venom's shipped config expects:

Model Memory (roughly) Role
qwen3:4b 4 GB the fast tier: subagents, summaries, titles
qwen3:8b 6 GB the main model on a laptop
qwen3:32b, qwen3-coder:30b 20 GB or more the main model on a workstation GPU

venom doctor --probe sends a tiny chat and a tool-call request to the model and reports when tool calls do not come back.

Ollama cuts prompts longer than its context length without saying so, and its default is small. Start it with a window that holds a coding session, and tell venom the same number so it compacts in time:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve
export VENOM_CONTEXT_WINDOW=32768

Ollama, direct

ollama pull qwen3:8b
ollama pull qwen3:4b
export VENOM_BASE_URL=http://localhost:11434/v1
venom doctor --probe
venom --offline

With no venom-* aliases on the server, venom maps them itself: the larger model serves venom-default and venom-smart, the smaller one venom-fast. When it cannot rank the models, set the map:

export VENOM_MODEL=qwen3:8b
export VENOM_ALIAS_MAP=venom-fast=qwen3:4b,venom-smart=qwen3:8b

--offline (or VENOM_OFFLINE=1) removes web_fetch and web_search from the tools and turns off remote MCP servers for the run, so the model is never offered a tool that would fail. Details: Local or cloud.

Inside a session, the local gear (key 4) switches to Ollama models; with an orchestrator on this machine, venom also moves a session to the local gear by itself when a cloud upstream stops answering (your first gear).

The orchestrator in front of Ollama

The orchestrator gives you the venom-default, venom-fast and venom-smart aliases, fallback between local models, session-sticky routing for Ollama's prompt cache, and a request log in SQLite. Its shipped config routes every alias to local engines first; cloud upstreams are disabled while their key variables are empty, so the same file runs off the grid.

Without Docker, the desktop daemon is the orchestrator plus a dashboard on 127.0.0.1:

curl -fsSL https://raw.githubusercontent.com/elishahbiz/venom-releases/main/install-daemon.sh | sh
venom-daemon status           # prints the /v1 URL and where the client key is
export VENOM_BASE_URL=http://127.0.0.1:8787/v1
export VENOM_API_KEY=…        # client_key in ~/.venom/daemon/secrets.json
venom --offline

Install it while you are online; afterwards it needs no network. Leave its auto-update switch and "Connect to Venom Cloud" off (the desktop daemon).

With Docker, from a source checkout, Ollama runs in a container next to the orchestrator:

make up SERVICES="orchestrator ollama"
make pull-models                  # qwen3:8b and qwen3:4b, the models the default routes use
export VENOM_API_KEY=…            # the VENOM_API_KEY value in .env (make up creates .env; leave the provider keys empty)
venom doctor
venom --offline

On macOS, Docker has no GPU: run Ollama natively (OLLAMA_CONTEXT_LENGTH=32768 ollama serve) and make up uses it instead of a container. make model picks and routes a larger local model, by hand or from what fits the GPU.

From the release binary, see the orchestrator quick start: the same config, started with VENOM_CONFIG, VENOM_API_KEY and VENOM_ADMIN_TOKEN.

The models it routes to are environment variables in the shipped config:

Variable Default Used by
OLLAMA_BASE_URL http://localhost:11434/v1 the ollama upstream
OLLAMA_CONTEXT_LENGTH 32768 the window the orchestrator assumes for Ollama; requests that do not fit fall back or get a 400 instead of being truncated
VENOM_OLLAMA_MODEL qwen3:8b venom-default, venom-smart
VENOM_OLLAMA_FAST_MODEL qwen3:4b venom-fast
VENOM_OLLAMA_EMBED_MODEL qwen3-embedding:0.6b venom-embed (semantic code search; local only)

To use llama.cpp or vLLM instead, add an upstream of kind llamacpp or vllm and put it in the routes (upstreams).

Air-gapped

For a machine that cannot download anything, build one archive on a connected machine and carry it over. This needs a source checkout:

make release VERSION=0.1.0
make airgap VERSION=0.1.0 AIRGAP_TARGETS="linux/x64"
# dist/airgap/0.1.0/venom-airgap-0.1.0-linux-x64.tar.gz and SHA256SUMS

The bundle holds the CLI with its own Node.js, the orchestrator, the shipped config and price table, the desktop daemon (AIRGAP_DAEMON=1, the default) and the harness libraries. It holds no model: AIRGAP_MODELS=<dir> copies a directory of model files you name into it, or you carry the models separately and compare their sha256sum.

On the target:

sha256sum -c SHA256SUMS
tar -xzf venom-airgap-0.1.0-linux-x64.tar.gz && cd venom-airgap-0.1.0-linux-x64
./install.sh --verify             # check every file, install nothing
./install.sh                      # into ~/.local (--prefix DIR, --daemon, --load-images)
venom-orchestrator >> ~/.local/share/venom/orchestrator.log 2>&1 &
export VENOM_API_KEY="$(cat ~/.local/share/venom/client-key)"
venom doctor
venom --offline

The installer needs no network, no Node.js and no root, and installs nothing when a check fails. With minisign installed it also verifies the release's signature; VENOM_REQUIRE_SIGNATURE=1 makes a missing signature check fatal. To bring an Ollama model, copy the models tree of a machine that ran ollama pull, or ollama create <name> -f Modelfile from a GGUF file.

Prove that nothing left

  • venom egress lists every host each session contacted (egress map). On an offline setup it should show only your model server.
  • venom doctor against a firewall rule that drops all outbound traffic: every check that matters passes.
  • From a source checkout, make test-airplane runs the test suites in a container with --network none after a separate container installed their dependencies, and fails if a test path makes an outbound call. It is how venom keeps this property from release to release.

What does not work offline

Web search and page fetch, remote MCP servers, cloud providers (their upstreams stay disabled and are never probed), Venom Cloud and mesh mode, and plugins from npm or git (copy a plugin directory in and run venom plugin add <path>). Everything else, the permission engine, subagents, checkpoints, code search and the request log included, works the same.

Troubleshooting

Symptom Cause and fix
The agent describes a tool call in text instead of making it The model does not emit structured tool calls. Use a qwen3 model; check with venom doctor --probe
Answers ignore the start of a long session Ollama truncated the prompt. Start it with OLLAMA_CONTEXT_LENGTH and set VENOM_CONTEXT_WINDOW to match
venom asks you to set the alias map The server lists more than 12 models or ids it cannot rank. Set VENOM_MODEL and VENOM_ALIAS_MAP
401 from the orchestrator Export the same VENOM_API_KEY the orchestrator has (.env, the daemon's secrets file, or client-key)
The first answer takes a minute Ollama is loading the model into memory. The shipped config gives Ollama 300 s to the first byte