---
title: Daemons, nodes, fleet, autoscaling and model lifecycle
description: Connect and run your own daemons, trust public nodes, see your org's agents, rent GPUs on demand and keep gears ahead of retiring models.
section: cloud
order: 86
audience: [user, operator]
sources: [services/nodejs-cloud/app/(console)/(shell)/daemons, services/nodejs-cloud/app/(console)/(shell)/join-tokens/page.tsx, services/nodejs-cloud/app/(console)/(shell)/nodes, services/nodejs-cloud/app/(console)/(shell)/fleet/page.tsx, services/nodejs-cloud/app/(console)/(shell)/autoscaling/page.tsx, services/nodejs-cloud/app/(console)/(shell)/lifecycle/page.tsx, services/nodejs-cloud/lib/autoscale, services/nodejs-cloud/lib/lifecycle, services/nodejs-cloud/lib/prewarm]
features: [cloud.console-daemons, cloud.console-join-tokens, cloud.console-nodes, cloud.console-fleet, cloud.console-autoscaling, cloud.console-lifecycle]
---

A daemon is the Go orchestrator in mesh mode on a machine you own: it dials out to Venom Cloud, advertises its models and serves requests routed to it, while its provider keys stay on the machine. These pages connect and run daemons, pick public nodes you trust, show the agents your org is running, rent GPU machines when requests wait, and warn before a model your gears use is retired.

| Page | Permission to view | What |
|---|---|---|
| `/daemons` | `daemons:read` | Your daemons as cards or a table, where requests go, prewarming. |
| `/daemons/new` | `daemons:read` (connect: `daemons:enroll`) | Connect a daemon: a join token, the command to run, a live wait for the first heartbeat. |
| `/daemons/{id}` | `daemons:read` (change: `daemons:manage`) | One daemon: status, controls, 24 h health and host metrics, models, prewarm, shares. |
| `/join-tokens` | `daemons:read` | Unused join tokens, to see and revoke. |
| `/nodes` | `routing:read` | The public node marketplace: nodes anyone can host, scored on what the cloud witnessed. |
| `/nodes/{id}` | `routing:read` | One public node: verification, quality score, uptime, models, prices, audit record, Trust. |
| `/fleet` | `logs:read` | The venom agents your machines are running, waiting ones first, and the session guard. |
| `/autoscaling` | `daemons:read` (change: `org:manage`) | GPU clouds, pools, machines, demand and the month's spend. |
| `/lifecycle` | `providers:read` (switch: `org:manage`) | Retiring models, price changes and prepaid provider credits. |

Roles: owners and admins have every permission here; developers can enroll and manage daemons and change routing; viewers read.

## Connect a daemon {#connect}

1. Open `/daemons/new` and pick the key group (private key groups first; a key group holds API keys, daemons, policies and GPU grants, not people). The page creates a join token (`vjt_…`) and shows the command for how you run the daemon, with this cloud's URL filled in.
2. Run it on the machine. The page offers several ways (the one-command `make daemon`, Docker, a release binary, the desktop app, plain env vars). With a release binary:

   ```bash
   # join once; paste the token when it waits (keeps it out of shell history)
   VENOM_MESH_NAME=gpu-box-1 venom-orchestrator mesh join https://cloud.usevenom.dev -
   # then run in mesh mode
   VENOM_CLOUD_URL=https://cloud.usevenom.dev VENOM_MESH_NAME=gpu-box-1 venom-orchestrator
   ```

   For systemd, Kubernetes or your own compose file, set `VENOM_CLOUD_URL`, `VENOM_MESH_JOIN_TOKEN` (first start only) and optionally `VENOM_MESH_NAME`, `VENOM_MESH_GEO`, `VENOM_MESH_ENDPOINTS`, `VENOM_MESH_GPUS` and `VENOM_MESH_MAX_JOBS` on the orchestrator.

3. The page waits for the first heartbeat and then links to the daemon.

The token is used once at enrollment; afterwards the daemon keeps its own identity. `/join-tokens` lists tokens not used yet, with prefix, key group, uses left and expiry, and **Revoke** behind a confirm. Creating a token on a plan with a daemon limit can answer `403 plan_limit_reached`.

## Your daemons {#daemons}

`/daemons` opens on a grid, one card per daemon: health and state in words, region, version, uptime, GPU utilisation now with a 24 h sparkline, VRAM and memory bars, jobs running and queued, tokens/s, temperature and power when the GPU reports them, the models Ollama holds warm and how many are cold, a "prewarming" badge, and for a public-pool daemon the credits earned today and in 30 days. A missing figure shows "–" with the reason on hover. `?view=table` switches to the table, with filters and bulk **Drain** and **Resume**.

Above the fleet, **Where requests go** sets the organisation's default routing: daemons then cloud, daemons only, or cloud only.

`/daemons/{id}` adds Drain, Resume, Revoke (type the name to confirm), Rename and labels; a 24 h health timeline (requests, errors and TTFT per hour, the last errors, admin events); the host over 24 hours in 5-minute points; models warm or cold; its prewarm; endpoints; shares with other key groups; and for a public-pool daemon its credits and claimed against witnessed quality of service.

All metrics come from the daemon's heartbeats; it sends nothing else. Metrics are kept 7 days.

### Prewarming {#prewarm}

A daemon can load a gear's local models before the first turn needs them:

| Trigger | What |
|---|---|
| A session starts on a gear | The CLI asks the cloud to warm the gear's models on the key group's live daemons that advertise them. Automatic. |
| Schedule (the Prewarm card on `/daemons`) | One per organisation: weekdays, a time and an IANA time zone, a gear (default: the org's default gear), and how long models stay loaded (5 to 720 minutes, default 120). Runs once within an hour after its time. |
| **Prewarm now** | On `/daemons` (every daemon) or a daemon's page (that one). |

A prewarm never evicts a model in use: an Ollama model loads only when it fits beside what is loaded; otherwise it is skipped with the figures. vLLM, SGLang and llama.cpp targets are always warm.

## Public nodes {#nodes}

`/nodes` lists nodes anyone can host on the public mesh, with what the cloud witnessed of them, scored on the server and filtered in the browser. It never shows an endpoint or anyone's private daemon. `/nodes/{id}` shows verification, the quality score and its parts, 24 hours of witnessed quality of service, uptime against the SLO over 24 hours, 7 and 30 days with the error budget's burn, models and prices (with provenance registry ids), the audit verdicts of ended epochs, and **Trust**, which adds the node to your organisation's allowlist. Region is self-reported. Trusted nodes serve only when your routing mode uses them; the page says so when it doesn't.

## Fleet {#fleet}

`/fleet` shows every venom agent your organisation's machines publish (every 15 seconds), grouped by machine, with agents waiting on a person first: state (`running`, `waiting`, `stuck`, `looping`, `idle`, `exited`), task, cost, tokens and file leases. Rows older than the time-to-live disappear. A machine whose user set `serve.public_url` links to its phone inbox. The page also shows the organisation's session guard: its limits and the sessions it paused. It refreshes every 15 seconds while visible.

## Autoscaling {#autoscaling}

Venom Cloud can rent GPU machines from your own RunPod or Lambda Cloud account when requests for your models wait, and return them when idle, within a monthly budget. Each machine boots the released daemon with Ollama and joins with a single-use join token. Nothing calls a GPU cloud until you connect one.

1. **Connect a GPU cloud** with your API key. The key is checked by listing what can be rented, sealed in the vault, and shown by its last four characters. **Refresh prices** lists again.
2. **Create a pool** with the guided form. The page turns it into one sentence, such as "Adds a machine when requests for qwen3:8b wait over 5 s for 3 min; removes it after 15 min idle; at most 2 machines, $300/month."

| Pool field | Notes |
|---|---|
| GPU type and count | From the cloud's listing, with prices. |
| Regions | Lambda: the region. RunPod: optional data centres. |
| Image | RunPod only. |
| SSH key name | Required by Lambda. |
| Models | Ollama models to pull. |
| Daemon release | Empty: latest. |
| `min` / `max` | Machines, at most 20. |
| `monthly_cap_usd` | No machine is added at or past it; `autoscale.cap_reached` is sent once a month. |
| Scale up | `wait_p90_s` (dispatch wait p90 in seconds) or `queue_depth`, held over `minutes`. Any refused request also counts. |
| Schedule | Optional: days, hours and time zone during which `keep` machines stay up. |
| `scale_down.idle_minutes` | A joined machine with no job for this long is drained. |
| `join_timeout_minutes` | A machine that hasn't joined by then is terminated. |

Per pool the page shows machines (state `requested`, `booting`, `joined`, `draining`, `terminated`, uptime, $/h, cost), this month's spend against the cap, and demand charts. **Scale to N now** holds a target for an hour; **Drain** and **Terminate** act on one machine. Each asks for confirmation. Spend is an estimate from the listed hourly price; your cloud's bill is the truth. Ollama is the only engine pools run today.

## Model lifecycle {#lifecycle}

`/lifecycle` ("Model lifecycle", under Usage & cost) lists models your gears, routing rows and allowlists use that are retired, retiring within the warning window, or gone from a provider's list, from a curated table shipped with the cloud plus your own catalog checks. The table can be late; a provider's own notice is the source of truth.

| Section | What you can do |
|---|---|
| Retiring models | See each model's state (`retired`, `soon`, `gone`), where it is used and a replacement. **Switch** rewrites every gear that names the model (audited, undoable); routing rows and allowlists are linked to change where they live. |
| Price changes | Models used in the last 30 days whose effective price changed, with the monthly impact. |
| Credits (`/lifecycle#credits`) | Prepaid balances or free quotas per provider key: amount, currency, start, optional expiry. Add, edit, remove (`providers:write`). **Refresh** reads the balance from DeepSeek, OpenRouter or Moonshot with the key's own secret, only when clicked. |
| Settings | Warning window (`warn_days`, default 30) and "use credits first" (`credits_first_days`, default 0 = off): keys with expiring credits move ahead of the org's other keys. |

Warnings arrive as bell notifications and the `model.retiring`, `price.changed` and `credits.expiring` webhook events. `venom doctor` also warns about gears naming a retiring model.

## Troubleshooting {#troubleshooting}

| Symptom | Cause and fix |
|---|---|
| `/daemons/new` keeps waiting | The daemon can't reach the cloud or the token was used or revoked. Check the daemon's log, then make a new token. |
| `403 plan_limit_reached` on a token | The plan's daemon limit. Upgrade under Settings, Billing, or revoke a daemon. |
| A daemon shows offline | Its lease lapsed. Check the machine; the `daemon.offline` webhook can tell you. |
| Pool machine ends `join_timeout` | It never enrolled: check the image, models and the cloud's console; raise `join_timeout_minutes` for large models. |
| Refresh on a credit answers `409 currency_mismatch` | The credit's currency differs from the provider's answer. Edit the credit. |

## Related

- [Self-hosting Venom Cloud](/docs/cloud/self-hosting.md): `make cloud-daemon` and `make cloud-sim` for local daemons.
- [Settings](/docs/cloud/console-settings.md): webhooks for daemon and autoscale events.
- [Venom Cloud overview](/docs/cloud.md) for the mesh and API pages.
