Docker Compose and profiles
Run the orchestrator and console with docker compose from a venom checkout, add local inference, metrics or web search with profiles, and share a GPU box with make daemon.
docker-compose.yml in a venom checkout holds every service. The Makefile wraps it, so you rarely type docker compose yourself. By default you get two containers, the orchestrator on 127.0.0.1:8787 and the console on 127.0.0.1:3000; everything else starts only when you name it.
Quick start
make env # .env from .env.example (mode 600) when missing
$EDITOR .env # set VENOM_ADMIN_TOKEN and VENOM_API_KEY; provider keys are optional
make up # orchestrator + console
curl -s http://127.0.0.1:8787/healthz
open http://127.0.0.1:3000 # sign in with VENOM_CONSOLE_PASSWORD, else VENOM_ADMIN_TOKENPoint the CLI at it with VENOM_BASE_URL=http://localhost:8787/v1 and VENOM_API_KEY set to the same client key.
The make targets
| Command | What it does |
|---|---|
make up |
Runs make down first, then docker compose up -d --build for SERVICES. Starts llmfit serve on the host (port 8789) when llmfit is installed, and points the orchestrator at a native Ollama on the host when one answers on :11434. |
make up SERVICES=orchestrator |
Only the gateway. SERVICES is any list of compose services, profiled ones included. |
make up SERVICES="orchestrator ollama" |
Fully local: the gateway plus a compose Ollama. No provider key needed. |
make up-mac |
macOS: orchestrator and console in Docker, Ollama native on the host so it gets the Apple GPU (Docker on macOS has none). Fails if no Ollama answers on :11434; start it with OLLAMA_CONTEXT_LENGTH=32768 ollama serve. |
make down |
Stops every compose service of every profile, plus this checkout's host-side processes (make dev-orchestrator, make dev-web, the mock, make serve, llmfit serve), matched by working directory so other checkouts keep running. Volumes are kept. |
make daemon |
Shares this machine's GPU in one go (below). |
make ps, make logs, make restart |
Status, logs and restarts for SERVICES. |
make model |
A menu (or BACKEND=ollama|vllm MODEL=…) that picks the local model, writes it to .env and routes the orchestrator to it. |
make dev-orchestrator, make dev-web |
The same programs on the host with hot reload, instead of in Docker. |
When .env sets VENOM_LOCAL_BACKEND (make model writes it), the default SERVICES becomes orchestrator web <backend>.
Services and profiles
A service under a profile only starts when you name it in SERVICES (or pass --profile to compose).
| Service | Profile | Host port | What it is |
|---|---|---|---|
orchestrator |
none | 127.0.0.1:${VENOM_PORT:-8787} |
The Go gateway. Config from ./config (read-only), data in a named volume at /data. |
web |
none | ${WEB_BIND:-127.0.0.1}:${WEB_PORT:-3000} |
The console (The web console). Starts after the orchestrator is healthy. |
ollama |
local |
127.0.0.1:11434 |
Ollama, CPU or NVIDIA GPU. Models in the ollama-models volume. |
llamacpp |
local |
127.0.0.1:8080 |
llama.cpp's llama-server, pulling LLAMACPP_HF (default Qwen/Qwen3-8B-GGUF:Q4_K_M) on first start. Uncomment its upstream in config/orchestrator.yaml. |
vllm |
gpu |
127.0.0.1:8000 |
vLLM (pinned v0.31.0) serving VLLM_MODEL (default Qwen/Qwen3-Coder-30B-A3B-Instruct). Reserves every NVIDIA GPU. |
prometheus, grafana |
observability |
127.0.0.1:9090, 127.0.0.1:3001 |
Metrics and a dashboard (Observability). |
searxng |
search |
127.0.0.1:${SEARXNG_PORT:-8888} |
Self-hosted metasearch, the first backend of the agent's web_search. It forwards queries to public search engines, so it is opt-in and not part of an offline setup. |
auditor |
audit |
127.0.0.1:9797 |
The public mesh's verifier; needs VENOM_CLOUD_URL, VENOM_CLOUD_AUDITOR_TOKEN and VENOM_AUDITOR_REFERENCES. |
postgres |
cloud |
127.0.0.1:5433 |
Venom Cloud's local database for cloud development, run as its own compose project, so make up and make down leave it alone. |
make up SERVICES="orchestrator web ollama" # local models
make up SERVICES="orchestrator vllm" # NVIDIA GPU server
make up SERVICES="orchestrator prometheus grafana" # + metrics
make up SERVICES="orchestrator web searxng" # + SearxNG for web_searchEvery port binds 127.0.0.1. Ollama, vLLM, llama.cpp and SearxNG have no auth of their own; the orchestrator reaches them over the compose network, and host tools over loopback. Only the console has a switch for another interface, WEB_BIND, because it has a password login.
NVIDIA GPUs. When docker info lists the nvidia runtime (Linux with nvidia-container-toolkit), the Makefile adds docker-compose.gpu.yml, which gives the compose Ollama every GPU. Without it Ollama runs on the CPU. The vllm service always reserves the GPU.
Where the orchestrator finds Ollama. Inside the container, localhost is the container itself, so make up sets OLLAMA_BASE_URL (via scripts/ollama-url.sh) to the compose ollama service when it is in SERVICES, else your OLLAMA_BASE_URL when it is not a loopback address, else a native Ollama on the host (http://host.docker.internal:11434/v1). Never put a localhost Ollama URL in .env for the Docker stack.
Dev and prod images
| Variable | Default | What it picks |
|---|---|---|
ORCH_TARGET |
dev |
The orchestrator's Dockerfile target. dev adds docker-compose.dev-orchestrator.yml, which mounts the source and hot-reloads with air. prod runs the built binary as the non-root venom user. |
WEB_TARGET |
dev |
The console's target. dev adds docker-compose.dev-web.yml (source mount, next dev); prod runs the standalone server.js. |
ORCH_TARGET=prod WEB_TARGET=prod make upBoth can live in .env too. The prod orchestrator uses its own data volume (orchestrator-prod-data), seeded from the image, so switching targets does not mix the two.
.env
make env copies .env.example to .env (gitignored, mode 600). The keys that matter for a deployment:
| Variable | Default | What it does |
|---|---|---|
VENOM_ADMIN_TOKEN |
change-me-admin in .env.example; dev-admin-token without a .env |
Bearer token for /admin/v1/*, and the console password when VENOM_CONSOLE_PASSWORD is empty. |
VENOM_API_KEY |
change-me-client; dev-venom-key |
The bootstrap client key for /v1/*. More keys come from the admin API. |
VENOM_CONSOLE_PASSWORD |
empty | The console login password (The web console). |
WEB_BIND |
127.0.0.1 |
Host address of the console port. 0.0.0.0 exposes it beyond this machine. |
WEB_PORT |
3000 |
Host port of the console. |
VENOM_PORT |
8787 |
Host port of the orchestrator. |
VENOM_SERVER_TOKEN |
empty | Lets the console's Agents pages reach venom serve on the host. |
OPENAI_API_KEY, ZAI_API_KEY, QWEN_API_KEY, ANTHROPIC_API_KEY, MISTRAL_API_KEY, MINIMAX_API_KEY, TYPESAFE_API_KEY |
empty | Provider keys. An empty key disables that upstream; nothing contacts the provider. |
OLLAMA_CONTEXT_LENGTH |
32768 |
Ollama's context window (its own default silently truncates at 4k on small GPUs). |
OLLAMA_KEEP_ALIVE, OLLAMA_NUM_PARALLEL, OLLAMA_MAX_LOADED_MODELS |
30m, 2, 2 |
Keep models and their cached prefix loaded. Use OLLAMA_NUM_PARALLEL=1 on a 16 GB machine. |
VLLM_MODEL, VLLM_TOOL_PARSER, VLLM_MAX_MODEL_LEN, VLLM_GPU_UTIL, VLLM_EXTRA_ARGS |
see .env.example |
The vLLM service's model and flags. The tool-call parser must match the model family. |
HF_TOKEN |
empty | Only for gated Hugging Face repositories. |
VENOM_CLOUD_URL, VENOM_MESH_NAME, VENOM_MESH_GEO, VENOM_MESH_ENDPOINTS |
empty | Mesh mode. Unset VENOM_CLOUD_URL means no mesh and no outbound connection. |
OTEL_EXPORTER_OTLP_ENDPOINT |
empty | Trace export to your own collector; empty is off. |
The full list, with every provider base URL and search key, is in Orchestrator environment variables.
Share a GPU box: make daemon
make daemon # stack + Ollama + recommended models
make daemon CLOUD_URL=https://cloud.usevenom.dev JOIN=vjt_… NAME=gpu-1 # and join Venom Cloudmake daemon (scripts/daemon.sh) does, in order:
- creates
.envwhen missing; - with
CLOUD_URL, writesVENOM_CLOUD_URLandVENOM_MESH_NAMEto.env, so every latermake upcomes back in mesh mode. The join token is never written anywhere: it goes only into the environment of the first start, asVENOM_MESH_JOIN_TOKEN, and the orchestrator keeps the credential it gets in its data volume; - runs
make upwith the orchestrator, the console and the compose Ollama (unless an Ollama already answers), with the GPU when Docker has one, andllmfit servefor hardware-fit estimates; - waits for
/healthz(300 s), then asks the orchestrator to pull and select the recommended default and fast models for this hardware; - prints the console's Local models page, the
/v1URL and the mesh state.
On a desktop without Docker, the desktop daemon does the same job.
Troubleshooting
| Symptom | Cause and fix |
|---|---|
make up-mac says no Ollama on :11434 |
Start Ollama natively: brew install ollama && OLLAMA_CONTEXT_LENGTH=32768 ollama serve. |
The orchestrator refuses to start on 0.0.0.0 |
It has no client key. Set VENOM_API_KEY (or VENOM_ALLOW_OPEN=1 if you really want an open gateway). |
The console exits at start with refusing to start |
A prod console needs VENOM_CONSOLE_PASSWORD or VENOM_ADMIN_TOKEN. |
| Local models answer with truncated context | Raise OLLAMA_CONTEXT_LENGTH; Ollama cuts silently past its window. |
| The vLLM container is unhealthy for a long time | The first start downloads the weights; its health check allows 60 minutes. |
| Agents pages say "not configured" | Set VENOM_SERVER_TOKEN and run venom serve on the host (The web console). |
Related
- The web console: the console's own environment.
- Observability: Prometheus and Grafana.
- Orchestrator configuration: upstreams and routes in
config/orchestrator.yaml. - Make targets: every target of the Makefile.