Run CI jobs in sandboxes - CostGraph Documentation

Running CI jobs in sandboxes is in preview.

Supported sandbox providers lists every provider that can run CI jobs. AutoPilot for CI can choose a sandbox as the runner for a job, next to GitHub-hosted runners and runner providers. A job sent to a sandbox runs on a GitHub Actions runner that takes only that job. The runner starts in a fresh sandbox in your provider account, exits when the job ends, and the sandbox is deleted. Sandbox runners compete like any other runner. They follow the workflow’s policy, trials and provider limits. Their cost is the provider’s published price for the sandbox size, for as long as the sandbox runs. Your provider bills that time to your account.

Requirements

AutoPilot for CI set up as described in AutoPilot for CI.
The CostGraph GitHub App with the organization permission Self-hosted runners set to Read and write, and the Workflow job event. An owner of the GitHub organization approves both in GitHub. Until then, sandboxes are skipped with reason github_app_reapproval.
A sandbox provider connected for CI, as described in the next section.
Linux jobs. Sandboxes run Linux only, on x64 or arm64.

Connect a sandbox provider for CI

Sandboxes are started by a dispatcher that holds your provider credentials. Run it in one of two ways.

Your own dispatcher

Install the agent runtime chart with your provider and its credentials, and turn on CI runners:

dispatcher:
  workerName: ci-fly
  existingSecret: costgraph-sandbox-key
  sandbox:
    provider: fly
    providerSecret: sandbox-provider
  ciRunner:
    enabled: true
    arch: x86_64

The provider credentials stay in your cluster. dispatcher.concurrency is how many CI jobs the dispatcher runs at once. Sandbox runners are a lean Ubuntu 24.04 image, not GitHub’s full ubuntu-latest image. They include a runner user with passwordless sudo, git, curl, a tool cache for the actions/setup-* actions, and Docker with buildx and compose. Install anything else in the job, with a setup-* action (actions/setup-node, setup-python, setup-go, setup-java) or sudo apt-get install -y <package>. Set dispatcher.ciRunner.providerTemplate to the CostGraph runner image (ghcr.io/baselinehq/costgraph-sandbox-runner) or a template built from it so jobs start faster. Without one, the sandbox starts from the provider’s default image and installs the runner when it boots. Fly Machines and Sprites run only through your own dispatcher.

CostGraph’s dispatcher

For a provider you connected to CostGraph for cost data, such as Microsandbox, CostGraph can start the sandboxes for you with that connection’s credential. The credential is read for each job and is not kept by the dispatcher.

Send a job to a sandbox

There are two ways to let AutoPilot run a job in a sandbox.

The routing job

Leave the routing job as it is. Sandboxes become candidates for every job that fits them, and the routing job’s log lists them with their reasons. A job that AutoPilot sends to a sandbox gets a runner label that starts with costgraph=.

A CostGraph label

Write a CostGraph label in runs-on to run a job in a sandbox without a routing job:

jobs:
  test:
    runs-on: costgraph=${{ github.run_id }}-${{ strategy.job-index }}/cpu=4/ram=16
    steps:
      - uses: actions/checkout@v5
      - run: make test

The label starts with costgraph= and a value unique to the job. The optional parts after it say what the job needs.

Part	Meaning
`cpu=<n>`	At least `n` vCPUs.
`ram=<gb>`	At least this many GB of memory.
`arch=x64` or `arch=arm64`	The CPU architecture.
`docker`	The job runs Docker. See Docker.

Only sandboxes compete for a job with a CostGraph label. AutoPilot runs the job on the best one under the workflow’s policy. To compare them with a runner you know, set a baseline runner.

What happens when a runner does not start

A runner that has not started 60 seconds after its sandbox was requested is tried once on the next best sandbox.
A job with a CostGraph label that still has no runner after 2 minutes gets a runner on the next best sandbox.
After the fallback delay, 300 seconds by default, a job with a CostGraph label that still has no runner goes to the fallback runner, when you set one.
A sandbox older than the job timeout of 6 hours is deleted.

AutoPilot checks for these every minute. A job routed by the routing job that never gets a runner is handled like any stuck job.

Settings

Set these for the organization, a repository or a workflow on the AutoPilot CI settings page, or with the settings API.

Setting	Meaning
Fallback runner	The connected sandbox a job with a CostGraph label goes to when no other sandbox started it.
Fallback delay	Seconds a job with a CostGraph label waits before it goes to the fallback runner. At least 60, 300 by default.
Baseline runner	The runner jobs with a CostGraph label are compared with for policy, trials and savings.

Secrets

A sandbox runs in your own provider account, so jobs with secrets can run there by default. To keep jobs that use secrets off a sandbox provider, set allow_secrets: false for it. Jobs then go there only when the routing job says they use no secrets with secrets: false.

Pull requests from forks

CostGraph never starts a sandbox runner for a run triggered by a pull request from a fork, or by pull_request_target. A fork run that has a job with a CostGraph label is cancelled, with a message that says why. Use the routing job in workflows that pull requests from forks trigger. It gives their jobs their default runners.

Public repositories

Sandbox runners join your organization’s Default runner group. By default, GitHub doesn’t let that group run jobs from public repositories. A job from a public repository then waits for a runner that never takes it. To allow them, an organization owner opens Settings > Actions > Runner groups in GitHub and turns on Allow public repositories for the Default group. CostGraph does not change this setting. Read GitHub’s guidance on self-hosted runners and public repositories before you turn it on.

Docker

Docker, including services: and container: jobs, works on sandbox runners where the provider can run a Docker daemon. CostGraph starts Docker before the job when the job needs it: the routing job’s docker input or docker in its CostGraph label. When that is not set, Docker is started anyway. A provider that cannot run Docker is not offered for jobs that need it.

Network

The runner connects out to GitHub over HTTPS on port 443. It opens no inbound port. If your provider limits egress, allow the domains GitHub lists for self-hosted runners.

​Requirements

​Connect a sandbox provider for CI

​Your own dispatcher

​CostGraph’s dispatcher

​Send a job to a sandbox

​The routing job

​A CostGraph label

​What happens when a runner does not start

​Settings

​Secrets

​Pull requests from forks

​Public repositories

​Docker

​Network

​Related

Requirements

Connect a sandbox provider for CI

Your own dispatcher

CostGraph’s dispatcher

Send a job to a sandbox

The routing job

A CostGraph label

What happens when a runner does not start

Settings

Secrets

Pull requests from forks

Public repositories

Docker

Network

Related