Skip to main content
Review NVIDIA GPU usage and estimated costs across Kubernetes clusters, Linux virtual machines, and bare-metal hosts. Use the inventory to find idle cards and check which workloads hold them.

Review GPU usage and costs

Open GPUs in the sidebar for the full inventory, or select the GPUs tab on a cluster or virtual machine to narrow the view.
  1. Filter by model, utilization, reporting time, or partitioning to find cards to review.
  2. Select a card. Overview shows its location, estimated cost, and any detected workloads or Multi-Instance GPU (MIG) partitions.
  3. Open Metrics for utilization, memory, power, and health charts. Use Usage to compare the latest measurements with 7-day and 30-day summaries.

Investigate idle or constrained GPUs

Use the utilization verdict to prioritize cards, then review usage history before changing capacity. For partial or stale data, check the exporter and recent readings before acting. CostGraph recommends changing a container’s GPU count only when its GPUs are exclusive to that container.

Share whole GPUs with MIG slices

Enable per-process GPU attribution to identify workloads whose peak memory fits a smaller slice of their GPU. In Insights, open a GPU sharing recommendation to review the affected workloads, target slice, estimated savings, and partition-change plan. Recommendations also identify pods waiting for a slice missing from the node’s layout. Available recommendations depend on the GPU model and cloud. Changing the partition layout evicts every pod on the node. Upgrade the NVIDIA driver first if the plan requires it. On supported clouds, AutoPilot applies the plan after your team approves it. Refer to GPU partition changes for cloud support and approval requirements.

Understand GPU cost estimates

GPU costs use catalog prices, not invoice totals.
  • Month-to-date cost estimates the observed hours in the current UTC month.
  • Projected monthly cost estimates a full month at the current rate.
  • Unpriced cards have no cost estimate and are excluded from the fleet spend total.
CostGraph uses a separate GPU rate when available, or derives one from the machine’s price. It counts physical cards without counting MIG slices or time-shared devices as additional GPUs. For workload-level charges, see GPU cost.

Set up GPU collection

Each GPU host needs the NVIDIA driver and DCGM exporter. Choose the setup for your environment, then return to GPUs to verify that cards report usage.

Kubernetes

Enable the exporter and its scrape target in your operator values.
Use device-name on GKE. For the standard NVIDIA device plugin, such as on EKS, use uuid instead. The wrong value prevents CostGraph from linking cards to their pods. Apply the values with helm upgrade. See Accelerators for profiling and per-process attribution, or Bundled exporters for selectors and tolerations.

Virtual machines and bare metal

Run the agent install script on a Linux GPU host. When nvidia-smi is available and the host uses systemd, the script also installs DCGM exporter. Rerun the script if the agent is already installed. For an existing exporter, set COSTGRAPH_GPU_EXPORTER_ENDPOINT to its metrics URL when running the installer. With Docker or without systemd, run the exporter separately and set COSTGRAPH_DCGM_ENDPOINT on the agent.

Fix missing GPU data

Check the symptom to distinguish collection failures from unsupported metrics.
Confirm nvidia-smi lists the card and DCGM exporter is running on its host. On Kubernetes, check both exporter settings and its pod’s placement. On a systemd host, run systemctl status nvidia-dcgm-exporter.
Check DCGM_EXPORTER_KUBERNETES_GPU_ID_TYPE against your device plugin, then apply the corrected operator values.
The GPU or driver does not publish that metric. Other available metrics remain usable. Refer to Measurement availability.