---
title: Build on Venom Experiments
description: Define metrics, propose, start, ramp, read and decide A/B tests and guarded rollouts on Venom Flags flags, run product analytics queries and send events, over the Experiments REST API, the event endpoint or MCP.
section: developers
order: 140
audience: [builder, agent]
sources: [services/nodejs-cloud/lib/analytics/mcp.ts, services/nodejs-cloud/lib/analytics/http.ts, services/nodejs-cloud/lib/analytics/service.ts, services/nodejs-cloud/lib/analytics/ingest.ts, services/nodejs-cloud/lib/flags/auth.ts, services/nodejs-cloud/app/experiments/api/v1, services/nodejs-cloud/app/experiments/sdk, services/nodejs-cloud/app/experiments/mcp, services/nodejs-cloud/contracts/experiments-openapi.yaml, docs/adr/0054-venom-experiments.md]
features: [cloud.experiments]
---

Venom Experiments runs A/B tests and guarded rollouts on [Venom Flags](/docs/developers/flags.md) flags. The flag's rollout assigns
units, so there is no second assignment system. Starting, ramping, stopping and deciding are Flags changes made as the
caller: they appear in the flag's history, and `flags_flag_revert` undoes them. Everything the app does is a REST
operation under `/experiments/api/v1` and an MCP tool. Your apps send events and exposures to
`/experiments/sdk/events` with a Flags environment key.

## What you can build {#build}

- **Experiments from your planning tool.** Propose a draft on a flag (`POST /experiments/api/v1/experiments`) with a hypothesis, a primary metric and guardrails; the answer has the planned sample and duration.
- **An agent that watches results.** Connect `venom-experiments` and have an agent call `exp_experiment_results` each day and `exp_experiment_decide` when the recommendation is clear.
- **Guarded rollouts in a release pipeline.** Propose with `kind: "rollout"` and `steps`, start it, and let the sweep ramp the flag while the guardrails hold; pause or resume it with `POST /experiments/api/v1/experiments/{id}/pause` and `/resume`.
- **Product analytics in your own dashboard.** Run a trend, funnel or retention query with `POST /experiments/api/v1/query` (or `exp_query_run`), or save it as an insight (`exp_insight_save`) that a Notes page embeds.
- **A decision log.** Subscribe an org webhook to `experiments.decided` and copy each decision, with its note and the flag's audit id, into your own records.
- **Event tracking from a backend.** Send up to 1000 events and 1000 exposures a request to `POST /experiments/sdk/events` with the environment's server SDK key.

## At a glance {#glance}

Generated from the code on every docs build.

| | |
|---|---|
| MCP endpoint | `https://cloud.usevenom.dev/workspace/mcp/exp`, server name `venom-experiments` |
| Connect | `claude mcp add --transport http venom-experiments https://cloud.usevenom.dev/workspace/mcp/exp` |
| MCP tools | 25 (12 read, 13 write) |
| OAuth scopes | `experiments:read` `experiments:write` |
| REST twin | none yet |
| Experiments API | 22 operations |
| Webhook events | `experiments.guardrail_breached`, `experiments.decided`, `experiments.started`, `experiments.ramped`, `experiments.stopped` |

The older endpoint `/experiments/mcp` still serves the same tools. `/exp/agents` in the app shows the connect
commands with your origin filled in.

## Authentication and scopes {#scopes}

An experiment is a way of changing a flag, so Experiments checks Flags' permissions: `flags:read` to read and
`flags:write` to change. They come from the same places as in Flags:

- A **personal API key** acts with its member's role: owners, admins and developers read and write; billing and viewer
  members read.
- A **team key** reads and writes.
- An **OAuth app** needs `experiments:read` (which stands for `flags:read`) and `experiments:write` (which stands for
  `flags:write`), within what the member's role allows. A token for `/workspace/mcp` or `/workspace/mcp/exp` works on
  those endpoints; only a token for the `experiments` resource (`/experiments/mcp`) works on `/experiments/api/v1`.
- The event endpoint `/experiments/sdk/events` takes an environment's server SDK key (`vfs_…`), mobile key (`vfm_…`)
  or client-side id (`vfc_…`), the same keys as [Flags' SDK endpoints](/docs/cloud/flags-sdk.md#keys).
- The org needs the Experiments add-on; otherwise each REST call answers `403 addon_required` and each tool says where
  to add it.
- Rate limits per credential: the REST API shares Flags' 600 requests a minute; `/experiments/mcp` allows 240 a
  minute. Past the limit you get `429` with `Retry-After`.

In an environment that requires approval, a start, ramp, stop or decide becomes a Flags approval request instead of
applying, and the answer says so (`flagChange.pendingApproval`).

## Worked example {#example}

Define a conversion metric, propose an experiment on the `new-checkout` flag, start it, then read its results:

```bash
BASE="$VENOM_CLOUD/experiments/api/v1"
H="Authorization: Bearer $VENOM_API_KEY"

curl -X POST "$BASE/metrics" -H "$H" -H 'Content-Type: application/json' \
  -d '{"environment": "production", "key": "purchase", "type": "conversion", "event": "checkout_completed"}'

curl -X POST "$BASE/experiments" -H "$H" -H 'Content-Type: application/json' -H 'Idempotency-Key: exp-checkout-1' \
  -d '{"flag": "new-checkout", "primary": "purchase", "hypothesis": "A one-page checkout converts better.",
       "guardrails": [{"metric": "checkout_errors", "tolerated": 0.02}]}'

curl -X POST "$BASE/experiments/<experiment-id>/start" -H "$H" -H 'Content-Type: application/json' -H 'If-Match: "1"' \
  -d '{"turnOn": true, "comment": "50/50 in production"}'

curl "$BASE/experiments/<experiment-id>/results" -H "$H"
```

The guardrail metric must exist too. Start, ramp, pause, resume, stop and decide answer
`{experiment, flagChange: {auditId, diff, pendingApproval, dryRun}, warnings}`; `flagChange.auditId` is the Flags
history entry to revert. Each answer's `ETag` is the experiment's version.

The same proposal over MCP, from any client connected to `/workspace/mcp/exp`:

```json
{ "jsonrpc": "2.0", "id": 1, "method": "tools/call",
  "params": { "name": "exp_experiment_propose",
              "arguments": { "flag": "new-checkout", "primary": "purchase",
                             "hypothesis": "A one-page checkout converts better.",
                             "guardrails": [{ "metric": "checkout_errors", "tolerated": 0.02 }],
                             "idempotencyKey": "exp-checkout-1" } } }
```

Then `exp_experiment_start` with `{"id": "<experiment-id>", "turnOn": true, "expectVersion": 1}` and
`exp_experiment_results` with `{"id": "<experiment-id>"}`.

## Running an experiment {#run}

A typical run for an agent:

1. `exp_metric_list`, then `exp_metric_create` for what is missing (`count`, `sum`, `conversion`, `ratio` or
   `retention` over tracked events; `direction: "down"` when less is better).
2. `exp_experiment_propose` with the flag, the primary metric, a hypothesis and guardrails. `exp_plan_get` (or
   `GET /experiments/api/v1/plan?primary=…`) estimates the sample alone, saving nothing.
3. `exp_experiment_start`. It is refused when the flag is off in the environment unless `turnOn` is true.
4. `exp_experiment_results` each day. `recommendation.action` is `ship`, `rollback`, `keep_running`, `inconclusive`
   or `investigate`. `investigate` means a sample ratio mismatch: do not act on the numbers.
5. `exp_experiment_decide` with `decision` (`ship` with a `variation` index, `rollback` or `iterate`) and a `reason`.
   It applies the Flags change, writes a decision note into Notes and sends `experiments.decided`.

Writes take `expectVersion` (or `If-Match`), `idempotencyKey` (or `Idempotency-Key`; a retry within 24 hours returns
the first answer), `comment` (shown in the flag's history) and `dryRun`. Shares are in thousandths of a percent
(50000 = 50 %), between 1000 and 99000. Ramping restarts the analysis, so ramp early and rarely. A metric an experiment
uses can only be renamed.

A guarded rollout (`kind: "rollout"`) needs at least one guardrail. Its default ladder is 1 %, 5 %, 25 %, 50 %, then
100 %, a day each. The sweep advances it, promotes it after the last step, or rolls it back and files a Tasks issue
when a guardrail breaks. `exp_experiment_rollout` (`action: "pause"` or `"resume"`) holds it; a breach still rolls a
paused rollout back.

## Sending events {#ingest}

`POST /experiments/sdk/events` takes `{"events": [...], "exposures": [...]}`: up to 1000 of each, 512 KB. An event is
`{event, unit: {kind, key}, value?, props?, ts?}`; `props` is at most 2 KB of JSON. Invalid entries are dropped and
counted in the `202` answer. Past the org's daily cap it answers `429` with `Retry-After`. See
[Sending events](/docs/cloud/experiments-events.md) for the TypeScript SDK's `track()` and retention.

## Events {#events}

5 org webhook events, generated from `lib/webhooks/events.ts`. Subscribe an endpoint to them in Settings, Webhooks (see [Webhooks and events](/docs/developers/webhooks)).

| Event | When |
|---|---|
| `experiments.guardrail_breached` | A guardrail metric of a running experiment is worse than its tolerated regression (once per experiment and metric). |
| `experiments.decided` | An experiment was decided: a variation shipped, a rollback or an iteration, by a person, an agent or the automatic guardrail rollback. |
| `experiments.started` | An experiment started serving its treatments (by a person, an agent, or an approved request): the experiment, flag, environment, the treatments' share and who started it. |
| `experiments.ramped` | A running experiment's treatment share changed (by hand, or a guarded rollout's next step): the share before and after, and who. |
| `experiments.stopped` | A running experiment was stopped and control is served to everyone: the experiment, flag, environment and who stopped it. |

The data of both events has the experiment's `experiment` id, `name`, `flag`, `project`, `environment` and `url`.
`experiments.decided` adds `decision`, `variation`, `reason`, `note` (the decision note's path) and `auditId` (the flag's
history entry); an automatic rollback adds `automatic: true`. `experiments.guardrail_breached` adds `metric`, `arm`,
`harm`, `tolerated`, `p` and `autoRollback`, and is sent once per experiment and metric.

Experiments has no webhooks of its own. The flag changes it makes also reach [Flags webhooks](/docs/developers/flags.md#events) as
`flag.update`.

## REST routes {#rest}

The Experiments API: 22 operations, generated from `services/nodejs-cloud/contracts/experiments-openapi.yaml`. "Key or token" is a Venom API key or an OAuth access token; SDK keys and client ids belong to an environment and are for your app's runtime.

**/experiments**

| Method | Path | Summary | Auth |
|---|---|---|---|
| POST | `/experiments/sdk/events` | Send events and exposures | SDK key or client id |
| GET | `/experiments/api/v1/experiments` | List experiments | key or token |
| POST | `/experiments/api/v1/experiments` | Propose an experiment (a draft; nothing is served yet) | key or token |
| GET | `/experiments/api/v1/experiments/{id}` | Get an experiment | key or token |
| PATCH | `/experiments/api/v1/experiments/{id}` | Edit a draft | key or token |
| GET | `/experiments/api/v1/experiments/{id}/results` | Results from the daily rollups, cached until they change | key or token |
| GET | `/experiments/api/v1/experiments/{id}/timeline` | The primary metric's cumulative lift by day, for a chart | key or token |
| POST | `/experiments/api/v1/experiments/{id}/{action}` | Start, ramp, stop or decide an experiment (each is a Flags change as the caller) | key or token |

**/events**

| Method | Path | Summary | Auth |
|---|---|---|---|
| GET | `/experiments/api/v1/events` | The events explorer - event names with 14 days of volume, and a sample of recent events | key or token |

**/plan**

| Method | Path | Summary | Auth |
|---|---|---|---|
| GET | `/experiments/api/v1/plan` | Plan an experiment - the sample it needs and how long recent traffic takes to deliver it (nothing is saved) | key or token |

**/query**

| Method | Path | Summary | Auth |
|---|---|---|---|
| POST | `/experiments/api/v1/query` | Run a trend, funnel or retention query (read-only, time-limited; nothing is saved) | key or token |

**/insights**

| Method | Path | Summary | Auth |
|---|---|---|---|
| GET | `/experiments/api/v1/insights` | List saved insights of an environment | key or token |
| POST | `/experiments/api/v1/insights` | Save a query as an insight | key or token |
| GET | `/experiments/api/v1/insights/{id}` | One insight | key or token |
| PATCH | `/experiments/api/v1/insights/{id}` | Rename an insight or change its query (If-Match with its version) | key or token |
| DELETE | `/experiments/api/v1/insights/{id}` | Delete an insight (Notes pages that embed it show "Insight unavailable") | key or token |
| GET | `/experiments/api/v1/insights/{id}/result` | Run a saved insight now - what a Notes page embedding it shows | key or token |

**/metrics**

| Method | Path | Summary | Auth |
|---|---|---|---|
| GET | `/experiments/api/v1/metrics` | List metrics of an environment | key or token |
| POST | `/experiments/api/v1/metrics` | Create a metric | key or token |
| GET | `/experiments/api/v1/metrics/{key}` | Get a metric | key or token |
| PATCH | `/experiments/api/v1/metrics/{key}` | Update a metric (not what it measures while an experiment uses it) | key or token |
| DELETE | `/experiments/api/v1/metrics/{key}` | Delete a metric no experiment uses | key or token |

## MCP tools {#tools}

25 tools: 12 read, 13 write. Generated from the code; "write (destructive)" is the tool's `destructiveHint`. A tool with a REST route is the same call over HTTP.

| Tool | Title | Access | Scope | REST route |
|---|---|---|---|---|
| `exp_event_list` | List events | read | `experiments:read` |  |
| `exp_experiment_decide` | Decide an experiment | write | `experiments:write` |  |
| `exp_experiment_get` | Get an experiment | read | `experiments:read` |  |
| `exp_experiment_list` | List experiments | read | `experiments:read` |  |
| `exp_experiment_propose` | Propose an experiment | write | `experiments:write` |  |
| `exp_experiment_ramp` | Ramp an experiment | write | `experiments:write` |  |
| `exp_experiment_results` | Experiment results | read | `experiments:read` |  |
| `exp_experiment_rollout` | Pause or resume a rollout | write | `experiments:write` |  |
| `exp_experiment_start` | Start an experiment | write | `experiments:write` |  |
| `exp_experiment_stop` | Stop an experiment | write | `experiments:write` |  |
| `exp_experiment_timeline` | Experiment lift over time | read | `experiments:read` |  |
| `exp_experiment_update` | Change a draft experiment | write | `experiments:write` |  |
| `exp_insight_delete` | Delete an insight | write (destructive) | `experiments:write` |  |
| `exp_insight_get` | Read an insight | read | `experiments:read` |  |
| `exp_insight_list` | List insights | read | `experiments:read` |  |
| `exp_insight_result` | Run an insight | read | `experiments:read` |  |
| `exp_insight_save` | Save an insight | write | `experiments:write` |  |
| `exp_insight_update` | Change an insight | write | `experiments:write` |  |
| `exp_metric_create` | Create a metric | write | `experiments:write` |  |
| `exp_metric_delete` | Delete a metric | write (destructive) | `experiments:write` |  |
| `exp_metric_get` | Read a metric | read | `experiments:read` |  |
| `exp_metric_list` | List metrics | read | `experiments:read` |  |
| `exp_metric_update` | Change a metric | write | `experiments:write` |  |
| `exp_plan_get` | Plan a sample size | read | `experiments:read` |  |
| `exp_query_run` | Run a trend, funnel or retention query | read | `experiments:read` |  |

Each tool takes `format: "json"` for a structured answer instead of compact markdown. `project` defaults to the org's
default Flags project and `environment` to `production`. The tool list is the same for every caller; a read-only
credential is refused in words when it calls a write tool.

## Related {#related}

- [Venom Experiments](/docs/cloud/experiments.md): metrics, the analysis, sample ratio mismatch and guarded rollouts.
- [Sending events](/docs/cloud/experiments-events.md) and [Insights and dashboards](/docs/cloud/experiments-insights.md).
- [Build on Venom Flags](/docs/developers/flags.md): the flags experiments run on, and how to revert a change.
- [Authentication and scopes](/docs/developers/auth.md), [Errors, limits and pagination](/docs/developers/conventions.md), [Webhooks and events](/docs/developers/webhooks.md) and [MCP](/docs/developers/mcp.md).
