> ## Documentation Index
> Fetch the complete documentation index at: https://docs.costgraph.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AutoPilot for sandboxes

> Send each new sandbox to the provider in your list that fits your policy, and see what every sandbox costs per agent, run, and customer.

AutoPilot for sandboxes chooses which of your sandbox providers starts each new
sandbox. It also records when each sandbox starts and stops. You can then see
what your sandboxes cost by provider and by the tags you set, such as agent,
run, or customer.

It works with [ComputeSDK](https://github.com/computesdk/computesdk), the open
source TypeScript SDK with one API for many sandbox providers. You wrap the
ComputeSDK providers you already use with `@costgraph/computesdk`, and every
other ComputeSDK call stays the same. From other languages, call the
[REST API](https://docs.costgraph.ai/costgraph/autopilot/sandboxes-api.md) directly.

You keep your own provider accounts and keys. Your sandboxes run in those
accounts, and each provider bills you as before. Your provider keys stay in
your process and are never sent to CostGraph.

## Requirements

* A CostGraph API key with the `sandbox:route` scope. Create it in
  **Settings > Personal API Keys**.
* Node.js 18 or later, or another runtime with a global `fetch`.
* `computesdk` 4.1.15 or later, and the ComputeSDK provider package for each
  provider you use, such as `@computesdk/e2b`.

## Set it up

**1. Store your API key**

Keep the key with the `sandbox:route` scope in your environment as
`COSTGRAPH_API_KEY`.

**2. Install the package**

Install `@costgraph/computesdk` next to ComputeSDK.

```shell theme={null}
npm install @costgraph/computesdk computesdk
```

**3. Wrap your providers**

Move your ComputeSDK providers inside `costgraph()`. The order of the list
matters. The first provider is the one your sandboxes use when AutoPilot
is off, and AutoPilot compares every other provider with it.

```ts theme={null}
import { compute } from 'computesdk';
import { modal } from '@computesdk/modal';
import { vercel } from '@computesdk/vercel';
import { costgraph } from '@costgraph/computesdk';

compute.setConfig({
  provider: costgraph({
    apiKey: process.env.COSTGRAPH_API_KEY,
    providers: [
      modal({ tokenId: MODAL_TOKEN_ID, tokenSecret: MODAL_TOKEN_SECRET }),
      vercel({ token: VERCEL_TOKEN, teamId: VERCEL_TEAM_ID, projectId: VERCEL_PROJECT_ID }),
    ],
  }),
});
```

Creating, using, and destroying sandboxes stays the same.

**4. Tag your sandboxes**

Pass tags in `metadata` when you create a sandbox. CostGraph groups cost by
these tags on the AutoPilot page.

```ts theme={null}
const sandbox = await compute.instances.create({
  sizeClass: 'md',
  metadata: { agent: 'researcher', run: runId, customer: customerId },
});
```

Providers that pass metadata on to the vendor keep doing so.

**5. Check that sandboxes arrive**

Create a sandbox, then open **AutoPilot** in CostGraph and select the
**Sandboxes** tab. The sandbox appears under **History** with its
provider, size, and tags.

At this point CostGraph tracks what every sandbox costs, and your sandboxes
still start on the first provider in your list. To let AutoPilot choose the
provider, turn routing on as described in
[Turn on routing](#turn-on-routing).

### Flush reports on shutdown

The package sends start and stop reports in the background, and its timers do
not keep your process running. Before your process exits, wait for queued
reports with `flush()`. It resolves within 10 seconds by default, or within the
limit you pass in milliseconds.

```ts theme={null}
const cg = costgraph({
  apiKey: process.env.COSTGRAPH_API_KEY,
  providers: [e2b({ apiKey: process.env.E2B_API_KEY })],
});
compute.setConfig({ provider: cg });

process.on('SIGTERM', async () => {
  await cg.flush(5_000);
  process.exit(0);
});
```

On edge runtimes such as Cloudflare Workers, background work stops when the
response is sent. Hand the flush to the runtime with `ctx.waitUntil(cg.flush())`.

### Package options

`costgraph()` accepts the following options.

| Option | Default | Description |
| - | - | - |
| `apiKey` | Required | Your CostGraph API key. If it is empty, sandboxes start in list order and nothing is reported. |
| `providers` | Required | Your ComputeSDK providers. Each must have a unique `name`. |
| `baseUrl` | `https://api.costgraph.ai` | The CostGraph API address. |
| `tenantId` | None | The tenant to report to, sent as the `X-CostGraph-Tenant-ID` header. |
| `routeTimeoutMs` | `300` | The longest a create waits for a routing answer, in milliseconds. |
| `cacheTtlMs` | `300000` | How long a routing answer is reused, in milliseconds. |
| `fetch` | Global `fetch` | A custom `fetch` implementation. |
| `onError` | Ignore | A function called with any CostGraph error, for logging. Errors never reach your sandbox calls. |
| `fallbackOnError` | `true` | Set to `false` to never try another provider after a failed create. ComputeSDK's own `fallbackOnError` setting does not reach a wrapped provider, so set it here. |

On a [self-hosted installation](https://docs.costgraph.ai/costgraph/self-hosted/overview.md), set `baseUrl`
to your installation's API address. Provider prices and start times reach your
installation with its pricing updates.

## Turn on routing

AutoPilot for sandboxes is off by default. While it is off, every sandbox
starts on the first provider in your list, and CostGraph still records what
each sandbox costs.

**1. Open the settings**

In CostGraph, open **AutoPilot**, select the **Sandboxes** tab, and select
**Settings**. Only admins can change these settings.

**2. Turn it on**

Turn on **Enabled**.

**3. Choose a routing policy**

Select a **Routing policy**. The default is **Cheapest**. The policies are
described in [Policies](#policies).

New sandboxes pick up a change within a few minutes, because the package
reuses each routing answer for up to five minutes.

## How a provider is chosen

When you create a sandbox with a `sizeClass`, the package asks CostGraph to
order your providers for that size. It then creates the sandbox on the first
provider in that order.

* **Your first provider is the baseline.** Every policy compares the other
  providers with it.
* **Only providers that support the size take part.** If none of your providers
  supports the size, `create()` throws the same `InstanceSizeError` that
  ComputeSDK throws without CostGraph. Refer to [Sizes](#sizes).
* **A sandbox without a size stays on your first provider.** Each provider has
  its own default size, so their prices are not comparable. CostGraph still
  records the sandbox and estimates its cost.
* **Your order is kept when AutoPilot can't compare.** This happens when
  AutoPilot is off, when the sandbox has no size, and when your first provider
  has no list price for the size.
* **Only the first 20 providers are priced.** A provider name CostGraph does not
  recognize stays in the order without a price.

### Expected cost and start time

AutoPilot compares providers on two numbers.

**Expected cost** is what one sandbox of the requested size costs for its
expected run time. The run time is rounded up to the provider's billing
increment, and is at least its minimum billed time, at the provider's list
price per hour. The expected run time is the median run time of your stopped
sandboxes of that size that started in the last 30 days, capped at 24 hours.
Until you have 20 such sandboxes, AutoPilot uses 10 minutes.

**Start time** is the median time from the create request to the first command
in the sandbox, in [ComputeSDK's public burst benchmark](https://github.com/computesdk/benchmarks),
which starts 100 sandboxes at once. The **success rate** is the share of those
100 starts that succeeded. A provider that the benchmark does not cover has no
start time and no success rate.

### Policies

Each policy orders your providers differently.

| Policy | Order |
| - | - |
| **Cheapest** | Providers with a price for the size come first, lowest expected cost first. A provider whose start time is more than twice the baseline's goes after the other priced providers. A missing start time never holds a provider back. |
| **Balanced** | Among providers that cost no more than the baseline and start within a limit, the best combination of saving and start time comes first. The baseline stays first when no provider improves on it. |
| **Fastest** | Among providers that cost no more than the baseline, the fastest to start comes first. The baseline stays first unless another provider starts strictly faster. |

Under **Balanced**, the limit is the baseline's start time plus 100ms or 10%,
whichever is larger. Each provider within the limit scores its saving on the
baseline's expected cost multiplied by how far its start time is under the
limit. Providers that score above zero come first, highest score first.

Every policy applies the following rules:

* A provider other than the baseline whose success rate is below 95% moves to
  the end.
* Providers without a price for the size come after the priced providers.
* Remaining ties keep your order.

The [REST API page](https://docs.costgraph.ai/costgraph/autopilot/sandboxes-api.md#choose-a-provider)
describes the full tie-break order for each policy.

### When CostGraph doesn't answer

Problems reaching CostGraph never make a sandbox create fail. A create waits up to
`routeTimeoutMs`, which is 300ms by default. If CostGraph is slow, unreachable,
or returns an error, the sandbox starts in the order of your `providers` list.
An answer that arrives after the timeout is kept for the next create. After a
failed call, creates use your list order for 30 seconds before asking again.

The package reuses an answer for `cacheTtlMs`, which is 5 minutes by default,
for each combination of size and provider list. Most creates make no extra
call.

### When a provider fails

If the chosen provider fails to create the sandbox, the package tries the next
provider only when ComputeSDK would. It does not try another provider in the
following cases:

* The create has a `sizeClass`.
* The create was aborted.
* The provider marks the error as not safe to retry.
* `fallbackOnError` is `false`.

Otherwise the package tries the next provider in CostGraph's order, then any of
your providers that the answer did not list.

A create from a snapshot, or with volumes that CostGraph saw created or listed,
goes to the provider that holds them.

### Choose the provider yourself

To skip routing for one sandbox, call the `costgraph()` provider directly and
name the provider. CostGraph still records the sandbox's start and stop.

```ts theme={null}
const sandbox = await cg.instances.create({
  provider: 'e2b',
  metadata: { agent: 'researcher' },
});
```

`compute.instances.create({ provider: 'e2b' })` does not work for this, because
`compute` knows only the `costgraph` provider.

## Sizes

`sizeClass` requests one of ComputeSDK's standard sizes.

| Size | vCPUs | Memory |
| - | - | - |
| `xs` | 1 | 1 GiB |
| `sm` | 1 | 2 GiB |
| `md` | 2 | 4 GiB |
| `lg` | 4 | 8 GiB |
| `xl` | 8 | 16 GiB |
| `2xl` | 16 | 32 GiB |

A provider supports a size in one of two ways:

* **Native.** The ComputeSDK provider package maps sizes itself, as Archil and
  Namespace do. The package passes the size on unchanged.
* **Mapped.** The provider package does not map sizes, but the provider accepts
  CPU and memory for each new sandbox. The package turns the size into that
  provider's own create options, following the provider's documented values.

The following providers take part in sized creates through mapping.

| Provider | Sizes | Create options |
| - | - | - |
| Beam | `xs` to `xl` | `cpu`, `memory` |
| Daytona | `xs` to `lg`, only with `image` | `resources.cpu`, `resources.memory` |
| Modal | `xs` to `2xl` | `cpu` and `cpuLimit` in physical cores of two vCPUs each, `memoryMiB` |
| Northflank | `xs` to `2xl` | `deploymentPlan`, from `nf-compute-100-1` to `nf-compute-1600-32` |
| Runloop | `xs` to `2xl` | `launch_parameters.resource_size_request`, with a custom size from `lg` up |
| Vercel | `xs` to `xl` | `resources.vcpus` |

Some providers have limits on the sizes they create:

* **Runloop and Vercel have no 1 vCPU, 1 GiB size.** For `xs` they create their
  smallest size, 1 vCPU with 2 GiB. They are used for `xs` only when none of your
  other providers creates exactly 1 vCPU with 1 GiB, and CostGraph records the
  size they created.
* **Daytona accepts CPU and memory only for sandboxes built from an image.** Pass
  `image`, such as `image: 'python:3.12'`, together with `sizeClass`. Without
  `image`, or with `snapshotId` or `templateId`, Daytona does not take part in
  sized creates. `xl` and `2xl` are above Daytona's default limit of 4 vCPUs and
  8 GiB.
* **Vercel allows 16 vCPUs only on Enterprise plans.** `2xl` is not sent to
  Vercel.

ComputeSDK does not allow CPU or memory options such as `cpu`, `memory`,
`resources`, `launch_parameters`, or `deploymentPlan` next to `sizeClass`.
Other options you pass are kept.

### E2B templates

E2B sets CPU and memory per template, not per sandbox, so E2B does not take part
in creates with `sizeClass`. To run E2B sandboxes at a given size, build a
template with that size with E2B's SDK. Then create sandboxes from it by name,
without `sizeClass`.

```ts theme={null}
await Template.build(template, 'agent-md', { cpuCount: 2, memoryMB: 4096 });

const sandbox = await cg.instances.create({
  provider: 'e2b',
  templateId: 'agent-md',
  metadata: { agent: 'researcher' },
});
```

CostGraph records these sandboxes without a size, so they are not routed or
compared by size.

### Sandboxes without a size

CostGraph prices a sandbox created without a size at its provider's documented
default size, when the provider's price list gives one. **History** labels it
**Default size**. A sandbox without a size is never moved to another provider.

## What CostGraph receives

The package sends only the following to CostGraph:

* Your CostGraph API key.
* The names of your providers, such as `e2b` and `daytona`.
* The size class of each create, or the vCPUs and memory a provider received
  when it created a larger size than requested.
* For each sandbox, the provider name, sandbox ID, start and stop times, and
  the ID of the routing answer.
* The `metadata` you pass to `create()`. Only string values are sent, with at
  most 20 keys. Keys can be up to 64 bytes, and values are cut to 256 bytes.

CostGraph never receives your provider keys, code, commands, environment
variables, files, or command output.

The package sends reports in batches of up to 100, at least every 2 seconds.
It retries a failed send a few times and then drops it. It queues at most
10,000 reports, and drops the oldest beyond that.

## Read the Sandboxes tab

Open **AutoPilot** and select the **Sandboxes** tab. The date range at the top
of the page sets the period. Until CostGraph receives its first sandbox,
**Managed** and **History** show the setup steps.

The tab opens with the following totals for the period.

| Tile | Meaning |
| - | - |
| **Sandboxes** | Sandboxes reported starting in the period. |
| **Estimated cost** | Each sandbox's run time, up to 24 hours, rounded up the way its provider bills, at the list price for its size. |
| **Saved** | What routed sandboxes would have cost on the first provider in your list, minus what they cost. |
| **Routed** | Sandboxes AutoPilot started on a different provider than the first one in your list. |
| **Running** | Sandboxes with no stop reported yet. Their cost grows until they stop, up to 24 hours. |
| **Unpriced** | Sandboxes with no list price for their provider or size. They add no cost. |

A sandbox counts toward **Saved** only when it started on the provider
AutoPilot chose, at the size it was routed for. Its saving is measured against
the first provider's own price, billing increment, and minimum billed time.

The tab has three views:

* **Managed** shows **Cost by provider**, with the sandboxes, run time, and
  estimated cost on each provider. Below it, **Cost by tag** lists the 50 tags
  with the highest estimated cost, from the `metadata` you pass when you create
  a sandbox.
* **Settings** has the **Enabled** switch and the **Routing policy**.
* **History** lists every sandbox reported in the period, newest first, with
  when it started, its provider, size, run time, estimated cost, and tags. A
  routed sandbox shows the provider it was routed from and what it would have
  cost there. A sandbox with no stop reported shows **Running**.

## Limits

* **A sandbox that ends without `destroy()` is not reported as stopped.** This
  includes a sandbox that the provider's own timeout stops. It shows as
  **Running**, and its estimated cost grows until the 24-hour cap. Call
  `destroy()` on every sandbox you finish with.
* **Run time counts at most 24 hours.** Estimated cost covers at most 24 hours
  of each sandbox, including a sandbox that is still running.
* **Costs are estimates from list prices.** They do not include discounts,
  credits, or included usage on your provider account. Your provider's bill is
  what you pay. To see billed sandbox spend, connect the provider on the
  [Integrations](https://docs.costgraph.ai/costgraph/integrations/overview.md) page where CostGraph supports
  it.
* **Prices and start times come from public sources.** Expected cost uses each
  provider's list price, and start time uses ComputeSDK's public benchmark. Your
  own sandboxes can start faster or slower.

## Related

* [Use AutoPilot for sandboxes from any language](https://docs.costgraph.ai/costgraph/autopilot/sandboxes-api.md)
* [AutoPilot for CI](https://docs.costgraph.ai/costgraph/autopilot/ci.md)
* [CostGraph AutoPilot for Kubernetes](https://docs.costgraph.ai/costgraph/autopilot/overview.md)
* [Roles and permissions](https://docs.costgraph.ai/costgraph/security/roles.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.