Skip to content
Docs menu

Terraform BYOD GPU

Rent a GPU box on RunPod or Lambda Cloud with venom's Terraform module, install the released venom-daemon with Ollama and your models on it, and join it to Venom Cloud.

View .md

deploy/terraform/byod-gpu/ in a venom checkout is a Terraform module that rents one GPU box on RunPod or Lambda Cloud, installs Ollama, your models and the released venom-daemon, and joins the box to Venom Cloud with a join token. The daemon only dials out, so the box needs no open port, and its provider keys stay on it.

What the box runs

The bootstrap (bootstrap.sh.tftpl, as root):

  1. installs Ollama with its official installer, then ollama pull for each of models;
  2. installs venom-daemon from the public releases with install-daemon.sh, which checks the archive against the release's SHA256SUMS (venom_version pins one);
  3. runs orchestrator mesh join <cloud_url> - with the token on stdin, once; the credential lands in ~/.venom/daemon/data/mesh/ and the token is not needed again;
  4. runs venom-daemon start --no-open with VENOM_CLOUD_URL set: a systemd unit (venom-daemon.service, survives reboots) on Lambda; on RunPod the pod's start command, which re-runs the bootstrap on every pod start and keeps the daemon's state, the models and the enrollment on the persistent volume at /workspace.

Use

You need Terraform 1.5 or newer, curl and jq on the machine you run it from, and the cloud's API key in your environment. The module has no providers: it drives the clouds' REST APIs through scripts/runpod.sh and scripts/lambda.sh from local-exec, so terraform init needs no registry and the API key never enters the plan or the state.

cd deploy/terraform/byod-gpu
export RUNPOD_API_KEY=…            # or LAMBDA_API_KEY=…
export TF_VAR_join_token=vjt_…     # Venom Cloud → Daemons → Connect a daemon
terraform init
terraform apply -var cloud=runpod -var name=gpu-4090-1
# terraform apply -var cloud=lambda -var lambda_ssh_key_name=my-key -var lambda_instance_type=gpu_1x_a100_sxm4

The daemon shows up under Daemons in Venom Cloud once the bootstrap finishes (Ollama, the model pulls and the daemon install take a few minutes). terraform output status_command prints the instance's state from the cloud's API. terraform destroy terminates it (an instance already gone is fine); the daemon then shows offline in Venom Cloud, where you can remove it.

As a module from your own configuration, with the module directory copied next to it:

module "gpu" {
  source              = "./modules/byod-gpu"
  cloud               = "lambda"
  name                = "lambda-a10-1"
  join_token          = var.join_token
  lambda_ssh_key_name = "my-key"
  models              = ["qwen3:8b", "qwen3:32b"]
}

Variables

Variable Default What it sets
cloud required runpod or lambda.
join_token required, sensitive A vjt_… join token; used once.
name venom-byod The instance name and the daemon's name in Venom Cloud.
cloud_url https://cloud.usevenom.dev The Venom Cloud to join.
venom_version latest The venom-daemon release to install.
models ["qwen3:8b"] Ollama models to pull.
runpod_gpu_type NVIDIA GeForce RTX 4090 RunPod GPU type id.
runpod_gpu_count 1
runpod_cloud_type SECURE Or COMMUNITY.
runpod_image ubuntu:24.04 The driver comes from the host; CUDA comes with Ollama.
runpod_volume_gb, runpod_container_disk_gb 100, 30
lambda_region us-east-1
lambda_instance_type gpu_1x_a10
lambda_ssh_key_name required for Lambda An SSH key in your Lambda account.

State and secrets

  • The instance id is kept in .venom-byod/<cloud>-<name>.json next to your configuration; the destroy step reads it. Keep that directory with your Terraform state.
  • The join token is a Terraform variable, never a resource attribute, so it is not in the state. It does reach the box: in the pod's environment on RunPod (visible in the RunPod console) and in the instance's user data on Lambda. Use a token for this box only and revoke it in Venom Cloud once the daemon has joined.
  • Changing a box setting (GPU, image, region, instance type, models, version) replaces the box.

Autoscaling instead

For boxes that come and go with demand, Venom Cloud's Autoscaling page does the same from the cloud side: you connect your RunPod or Lambda key once, and it starts and stops boxes within a monthly budget, each with its own single-use join token. It renders the same bootstrap as this module.