Skip to main content
vLLM is a model server, not a gateway. It serves models you host yourself and reports tokens per model, with no notion of a caller. See the upstream production metrics reference.

​
What CostGraph scrapes

Override the namespace or selector on the connection if you serve from somewhere else.

​
How the metrics map

Both become costgraph_gen_ai_usage_total, with the direction set by which metric the value came from: model_name becomes costgraph_model, and the unit is always tokens. vLLM emits only model_name and engine, so there is no host or customer label to carry.

​
Attribution is per workload

Per-request attribution isn’t available, and no configuration changes that. You have two options:
The annotation fills in a customer only where the gateway did not report one itself, so annotating a shared gateway doesn’t collapse its customers onto one sub-account.

​
Pricing a model you host

Because you serve the model, there is no vendor rate to read. CostGraph divides what the serving namespace cost that day by the tokens it served that day, so usage prices at what it actually cost you to run. Pin your own rate instead with a costgraph.ai/pricing-id.model label on the workload. See Pricing IDs.

AI gateways

Connecting, pricing, and sending usage yourself.