Solo.io agentgateway - CostGraph Documentation

agentgateway reports GenAI token usage from its data plane. See the upstream LLM observability reference.

What CostGraph scrapes


Namespace	none, you supply it
Pod selector	none, you supply it
Port and path	`15020` `/metrics`
Metrics	names starting `agentgateway_gen_ai_`

Token metrics come from the per-Gateway data plane, not the control plane, and those pods live in whichever namespace you created the Gateway in. An agentgateway connection must name the namespace and the pod label selector, or nothing can ever be scraped.

Read the labels off the running data plane pods and narrow the selector until it matches them and nothing else:

kubectl -n my-namespace get pods --show-labels

How the metric maps

agentgateway_gen_ai_client_token_usage becomes costgraph_gen_ai_usage_total:

agentgateway label	CostGraph label
`gen_ai_response_model`	`costgraph_model`
`gen_ai_token_type`	`costgraph_usage_kind`
`gen_ai_system`	`costgraph_host`
`customer`	`costgraph_sub_account`

The unit is always tokens.

Attribute usage to your customers

Add a metric attribute computed from the request. On Kubernetes this goes in an AgentgatewayPolicy. The value is a CEL expression, so the identifier can come from a header, a JWT claim, or anything else in the request.

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
  name: costgraph-customer-attribution
  namespace: my-gateway-namespace
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: my-gateway
  frontend:
    metrics:
      attributes:
        add:
          - name: customer
            expression: request.headers["x-costgraph-customer-id"]

A policy applies only where it is targeted, so it needs targetRefs or targetSelectors naming your Gateway. It must also live in that Gateway’s namespace, because a policy cannot reach across namespaces. If a policy already targets the Gateway, add the attributes block to that one instead of creating a second.

Group usage by agent run

Add a run attribute next to customer in the same policy. A run’s model cost then appears next to its sandbox cost on AutoPilot > Sandboxes > Runs.

spec:
  frontend:
    metrics:
      attributes:
        add:
          - name: customer
            expression: request.headers["x-costgraph-customer-id"]
          - name: run
            expression: request.headers["x-costgraph-run-id"]

Send the header x-costgraph-run-id on each request, set to the run tag of the run’s sandboxes. Refer to Group cost by run.

AI gateways

Connecting, pricing, and sending usage yourself.

​What CostGraph scrapes

​How the metric maps

​Attribute usage to your customers

​Group usage by agent run

AI gateways

What CostGraph scrapes

How the metric maps

Attribute usage to your customers

Group usage by agent run