Persistent volumes - CostGraph Documentation

CostGraph AutoPilot can grow a workload’s PersistentVolumeClaims (PVCs) before they run out of space. It acts on sustained growth, not on short spikes, and records every grow in the workload’s AutoPilot history as size X -> Y with the reason. AutoPilot only grows volumes. It never shrinks one, because Kubernetes and the cloud providers do not support shrinking a volume. Volumes with more space than they use are shown as insights with the saving instead.

Turn it on

Storage is off by default at three levels: the chart, the cluster and the workload.

Enable it on the cluster

helm upgrade costgraph-operator costgraph/costgraph-operator \
  --reuse-values \
  --set global.autopilot.enabled=true \
  --set global.autopilot.storage.enabled=true

global.autopilot.storage.enabled turns on the cluster storage switch and grants the operator permission to patch PersistentVolumeClaims. It is not granted otherwise. See Operator configuration.

Annotate a workload or a namespace

kubectl annotate statefulset postgres -n payments autopilot.costgraph.ai/storage=auto

A workload annotation wins over its namespace. Storage works without autopilot.costgraph.ai/mode, so you can protect volumes without letting AutoPilot change CPU or memory. mode: off and mode: initial do not stop grows. mode: disabled does.

The opt-in covers every PVC the workload’s pods mount. Each PVC is evaluated and grown on its own, including each StatefulSet replica’s PVC. A PVC mounted by several workloads is grown if any of them is opted in. The settings are listed in Annotations.

When a volume grows

A volume grows when one of these holds:

Both its 30-day and its 7-day growth trends say it will be full within 30 days. A one-off spike lifts the 7-day trend but barely moves the 30-day trend, so it does not trigger a grow.
It is at least 90% full, whatever the trend says.

A volume with less than 30 days of history is not grown on trend: it grows only once it is 90% full, and the history says why. Its recommendation is still shown. Grows are not limited to the change window. Usage data can be up to about 30 minutes old, so AutoPilot is meant for steady growth, not for a disk that fills in minutes.

How much it grows

The new size gives about 60 days of runway at the current growth rate. The recommendation CostGraph shows uses the same size.
A grow adds at least 10% and at least 1 GiB, so volumes are not grown in many small steps.
A volume that is 90% full grows at least enough to bring it back under 80% full.
A volume never grows past storage.max. Without it, the limit is 4x the volume’s size when AutoPilot first managed it. At the limit, grows stop, the history shows “growth limit reached: set storage.max to allow more”, and the near-full insight stays open.

Which volumes are managed

AutoPilot manages a PVC only when it can measure its usage and the storage driver can resize it. Every other volume is skipped, and the workload shows why.

Volume	What you see
Storage class without `allowVolumeExpansion: true`	Skipped with “storage class does not allow expansion”.
Driver that does not resize, even though its storage class allows expansion	The first grow on each storage class tests it. If the driver does not act, the grow is recorded as failed with “storage driver did not resize this volume” and that storage class is not grown again until it changes. Kubernetes does not let the request be lowered again, so it stays raised until the driver resizes the volume, and the history says so.
Driver that reports no usage, such as S3 and other object-store drivers	Not managed. Shown as “storage not measurable: driver reports no usage”.
Shared filesystems that report a nominal capacity, such as NFS or EFS	Not managed.
Statically provisioned PersistentVolume with no storage class	Not managed. Shown as “volume cannot be resized”.
`volumeMode: Block`	Not supported.

Missing usage is never read as empty or full. When every PVC of a workload is skipped, its chip shows storage as limited, with the reason. Some drivers expand the filesystem only when the pod restarts. AutoPilot does not restart pods for this. The history shows the grow as waiting for the filesystem resize until the pod restarts.

Provider limits

Cloud providers limit how often a volume can be changed. AutoPilot waits for the limit instead of sending a grow the cloud would reject, and the history shows when the next grow is allowed.

Driver	Limit
AWS EBS	4 changes per volume in any 24 hours. Each change must finish before the next.
Google Cloud Hyperdisk Balanced	2 changes per 4 hours.
Google Cloud Hyperdisk Throughput and Hyperdisk Extreme	1 change per 6 hours.
Other Google Cloud Hyperdisk types	1 change per 4 hours.
Google Cloud Persistent Disk (`pd-standard`, `pd-balanced`, `pd-ssd`)	No limit beyond `storage.cooldown`.
Azure Standard HDD, Standard SSD and Premium SSD up to 4 TiB	Cannot grow past 4 TiB while attached, so AutoPilot stops at 4 TiB. Premium SSD v2 and Ultra Disk are not capped.
Any other driver	1 grow per 6 hours.

storage.cooldown sets the minimum time between grows of one PVC. It defaults to 6h and is never shorter than the provider limit.

GitOps

AutoPilot grows a PVC even when its manifest lives in Git. Tell your GitOps tool to leave the size alone, or it reverts the grow on the next sync. The history entry for a grow on an Argo CD or Flux managed PVC carries the same reminder.

Argo CD
Flux

Add ignoreDifferences to the Application. RespectIgnoreDifferences=true makes Argo CD keep the live size during a sync as well as in the diff.

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  ignoreDifferences:
    - group: ""
      kind: PersistentVolumeClaim
      jsonPointers:
        - /spec/resources/requests/storage
  syncPolicy:
    syncOptions:
      - RespectIgnoreDifferences=true

With kustomize-controller v1.9.0 or later, add an ignore rule to the Kustomization.

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
spec:
  ignore:
    - paths:
        - /spec/resources/requests/storage
      target:
        kind: PersistentVolumeClaim

On older versions, annotate the PVC manifest in Git so Flux creates it once and does not update it afterwards.

metadata:
  annotations:
    kustomize.toolkit.fluxcd.io/ssa: IfNotPresent

A StatefulSet’s volumeClaimTemplates cannot be changed, so AutoPilot leaves them alone. Only existing PVCs grow. A new replica starts with the template’s original size, and AutoPilot grows it later if it needs more.

Pause and stop

These stop storage grows:

global.autopilot.paused=true or global.autopilot.storage.enabled=false on the operator chart, for the whole cluster.
autopilot.costgraph.ai/paused-until on a namespace or a workload, until that time.
autopilot.costgraph.ai/mode: disabled or autopilot.costgraph.ai/storage: off on a workload or a namespace.

A volume that is 90% full grows without waiting for storage.cooldown. It still waits for the provider limit and never grows while paused or disabled. AutoPilot never touches PVCs in kube-system or the CostGraph namespace.

​Turn it on

​When a volume grows

​How much it grows

​Which volumes are managed

​Provider limits

​GitOps

​Pause and stop