X-CostGraph-*
headers to outgoing requests. On Linux, CostGraph reads the headers without a
proxy and makes the attribution available in network cost views.
For example, X-CostGraph-Tenant: acme assigns the request’s bytes to the tenant
key with value acme. Use this identifier to compare traffic and costs by customer.
Requirements
Use a Linux node monitored by the CostGraph Operator or Agent. The client process must be visible through the host’s/proc and run in a container, unless you
enable host-flow collection.
Requests must use HTTP/1.1 or HTTP/2 over TCP and carry a non-empty attribution
header. HTTP/3 is unsupported. For HTTPS, use Go’s crypto/tls or dynamically
linked OpenSSL. Check Runtime compatibility for your client.
Set the header on the outgoing client request. A server receiving the header does
not attribute it. Any proxy hop you want to attribute must preserve the header.
macOS: Follow Request header attribution on macOS to configure the local proxy instead.
Enable capture
Per-request capture runs in the network metrics scraper and is on by default. Set the corresponding option tofalse to disable it.
- Operator
- Agent
Set While this is on, the
perRequestCollector explicitly in your operator values file if you need to
re-enable capture.flowtrace DaemonSet runs in the host PID namespace, so it can
see client processes. See
Network Metrics scraper
for its other settings.Host processes
By default, requests from processes that don’t belong to a container are dropped. To attribute traffic from processes running directly on the node, also setflowtrace.config.includeHostFlows to true in the operator chart, or pass
--flowtrace-include-host-flows to the agent.
Send tagged requests
Add one or moreX-CostGraph-<key> headers to the outgoing requests you want to
attribute. The part after X-CostGraph- is the tag key, and the header value is
the tag value.
- Go
- Python
- curl
Read the results
Open Network in the dashboard. Attributed requests are merged into the connection traffic on this page, so tagged and untagged traffic appear together. The cards at the top cover the selected period:- Network cost / mo is the monthly network cost, split into Internet, Cross zone, and Other. Its footer shows how many connections could be priced, and how cost and traffic divide between tagged and untagged requests.
- Top contributors lists the flows with the highest cost. Select one to open its connections.
Group and filter
- Group by adds one or more
tag:<key>dimensions. A request with two keys, liketenantandfeature, is counted under both, so totals across different keys can add up to more than the overall total. - Filter by narrows the results by Traffic (locality), Server kind, destination Port, or Tag value.
- The Connections table can be sorted by Cost or Traffic. Use Search, Min bytes, and Min opened to narrow it further, and Show all servers to remove the default limit on how many servers are listed. Expand a row to see the servers and connections behind it.
Locality
Each connection is priced by where its destination was:
A row that combines connections with different localities shows Mixed.
A cost of
0 means the traffic was measured and the provider doesn’t bill it.
A — means the traffic couldn’t be priced because CostGraph doesn’t know where
the peer is with enough precision.
How header names become keys
CostGraph normalizes header names before using them as attribution keys.
Because dashes become underscores,
X-CostGraph-Cost-Center and
X-CostGraph-Cost_Center produce the same key.
Limits
- 8 tag keys per request. Extra keys are not recorded.
- 256 bytes per value. Longer values are truncated. Invalid UTF-8 is replaced with the Unicode replacement character.
Proxies and sidecars
Attribution happens at the client process that sends the request. Behind a proxy, each hop that is itself a supported client can be attributed. This works only if the header is still on the request it forwards.- If your gateway or proxy strips unknown headers, allow
X-CostGraph-*through so later hops can attribute too. - A sidecar that originates TLS (for example Envoy in Istio, or the Linkerd proxy) uses a TLS stack that isn’t supported yet. The application’s own request to the sidecar can still be attributed if the application sends it through a supported client.
Runtime compatibility
CostGraph needs a TLS library it can hook. Node, Bun, Deno, and static or musl builds compile their TLS into the binary.
rustls, Java JSSE, and BoringSSL are TLS stacks CostGraph does not hook.
To check whether a binary links OpenSSL dynamically, run
ldd /path/to/binary | grep libssl. A libssl.so line in the output means the
library can be hooked.
Common problems
Requests do not appear
Requests do not appear
Check each item in Requirements, in this order:
- The header is set on the outgoing request from the client, not on the response or on the server.
- The client’s runtime is in the supported half of the compatibility table.
- Per-request capture is on for that node. It’s on by default, so check that
flowtrace.config.perRequestCollectoror--flowtrace-per-request-collectorisn’t set tofalse. See Enable capture. - The process runs in a container, or host-flow collection is on.
- The key contains only letters, digits, dashes, or underscores, and the value is not empty.
- The request completed. A request is attributed only when both its request and its response are captured, so requests that time out or are cancelled before a response arrives are not counted.
Only some hops are attributed
Only some hops are attributed
A proxy along the path is probably stripping the header, or the hop uses an
unsupported TLS stack such as a BoringSSL or rustls sidecar. See
Proxies and sidecars.
Separate keys were merged
Separate keys were merged
Dashes become underscores, so
X-CostGraph-Cost-Center and
X-CostGraph-Cost_Center share the key cost_center. Rename one of them.Cost shows as —
Cost shows as —
The traffic was measured, but CostGraph couldn’t place the destination well
enough to price it. This is usually a private address that doesn’t match a
known resource, or a peer in the same network whose zone isn’t known.