Amazon Web Services - CostGraph Documentation

CostGraph reads your AWS costs from a billing export that it sets up in your account. You start in CostGraph, finish in AWS, and your account connects when setup is done.

What the setup adds to your account

Read-only access to your billing data for CostGraph.
A storage bucket where AWS delivers your billing data.
Separate access for Workflows, if you let Workflows make changes.

Connect your account

In CostGraph, open Integrations, find AWS, and select Connect.
Choose a Tenant.
Choose what CostGraph can do. See Options.
Select Connect AWS. AWS opens in a new tab.
Select the checkbox that lets AWS create roles, then select Create stack.

Setup in AWS takes a few minutes. When it’s done, CostGraph shows AWS connected. You can close the CostGraph window while you wait. Your account appears under Connections when it’s ready. Your costs appear within about a day. The setup link works for one day. If it expires, select Connect AWS again. If your account already sends billing data to a bucket, enter that bucket’s name in ExistingExportBucket on the AWS page.

Options

Let Workflows make approved changes is on by default. Workflows only makes the changes your goals allow.
Also allow cleanup is on by default. Workflows can remove unused disks, backups, and IP addresses. You approve each one first.
Also allow launching capacity is off by default. Workflows can start servers, shut down the ones it started, and ask AWS for more capacity. This adds cost, and you approve each one first.
Activate cost allocation tags is on by default. You can group costs by your AWS tags. See Cost allocation tags.
Show AWS savings recommendations is on by default. See Savings recommendations.
Show how your resources connect is on by default. See Network access.

Change your options later

Update the CostGraph stack in AWS and change its settings. CostGraph picks up the change automatically. If you disconnect the account in CostGraph and want to connect it again, delete the CostGraph stack in AWS first. Your billing data stays in your account.

If your account doesn’t connect

CostGraph shows what went wrong. Fix the problem and select Connect AWS again.

Setup with Terraform

If you manage AWS in code, you can create the billing export with the following Terraform, then connect the account with the CostGraph API. The export is created in us-east-1, where AWS billing data lives, so aws_bcmdataexports_export must be managed by an AWS provider configured for that region. If your root provider targets another region, add an aliased one and point the export at it.

provider "aws" {
  alias  = "billing"
  region = "us-east-1"
}

data "aws_caller_identity" "current" {}

resource "aws_s3_bucket" "focus" {
  bucket = "costgraph-focus-export-${data.aws_caller_identity.current.account_id}"
}

resource "aws_s3_bucket_policy" "focus" {
  bucket = aws_s3_bucket.focus.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "EnableAWSDataExportsWrite"
      Effect    = "Allow"
      Principal = { Service = "bcm-data-exports.amazonaws.com" }
      Action    = ["s3:PutObject"]
      Resource  = ["${aws_s3_bucket.focus.arn}/*"]
      Condition = {
        StringLike   = { "aws:SourceArn" = "arn:aws:bcm-data-exports:us-east-1:${data.aws_caller_identity.current.account_id}:export/*" }
        StringEquals = { "aws:SourceAccount" = data.aws_caller_identity.current.account_id }
      }
    }]
  })
}

resource "aws_bcmdataexports_export" "focus" {
  provider = aws.billing

  export {
    name = "costgraph-focus-1-2"
    data_query { query_statement = "SELECT * FROM FOCUS_1_2_AWS" }
    destination_configurations {
      s3_destination {
        s3_bucket = aws_s3_bucket.focus.id
        s3_prefix = "focus-1-2"
        s3_region = "us-east-1"
        s3_output_configurations {
          output_type = "CUSTOM"
          format      = "PARQUET"
          compression = "PARQUET"
          overwrite   = "OVERWRITE_REPORT"
        }
      }
    }
    refresh_cadence { frequency = "SYNCHRONOUS" }
  }
}

First Parquet files land within about 24 hours. To also show AWS savings recommendations, add an inline policy named CostGraph-RecommendationsRead to the read role. Use your role’s name if it differs from CostGraph-Reader:

resource "aws_iam_role_policy" "recommendations" {
  name = "CostGraph-RecommendationsRead"
  role = "CostGraph-Reader"
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "cost-optimization-hub:ListRecommendations",
        "cost-optimization-hub:GetRecommendation",
        "cost-optimization-hub:ListRecommendationSummaries",
        "cost-optimization-hub:ListEnrollmentStatuses",
        "cost-optimization-hub:GetPreferences",
      ]
      Resource = "*"
    }]
  })
}

Then turn the AWS services on once, from the management account:

aws cost-optimization-hub update-enrollment-status --region us-east-1 \
  --status Active --include-member-accounts
aws compute-optimizer update-enrollment-status --region us-east-1 \
  --status Active --include-member-accounts

Read access

CostGraph reads the export bucket with a cross-account role granting s3:GetObject and s3:ListBucket on the export prefix, plus the cost allocation tag permission described below. CostGraph sets this up for you when you connect. With savings recommendations on, the role also gets read-only access to AWS’s recommendations:

cost-optimization-hub:ListRecommendations
cost-optimization-hub:GetRecommendation
cost-optimization-hub:ListRecommendationSummaries
cost-optimization-hub:ListEnrollmentStatuses
cost-optimization-hub:GetPreferences

With network access on, the role also gets a read-only CostGraph-NetworkRead policy. It reads your network layout, and the managed databases, caches, and shared file systems your workloads talk to, so Context can name them:

ec2:DescribeNatGateways
ec2:DescribeNetworkInterfaces
ec2:DescribeRegions
ec2:DescribeRouteTables
ec2:DescribeSubnets
ec2:DescribeVolumes
ec2:DescribeVpcEndpoints
eks:DescribeCluster
eks:ListClusters
elasticache:DescribeCacheClusters
elasticache:DescribeReplicationGroups
elasticfilesystem:DescribeFileSystems
elasticfilesystem:DescribeMountTargets
elasticloadbalancing:DescribeLoadBalancers
elasticloadbalancing:DescribeTargetGroups
elasticloadbalancing:DescribeTargetHealth
rds:DescribeDBInstances

Cost allocation tags

AWS only includes a tag in billing data once the tag is active for cost allocation. After you connect, and on each sync, CostGraph activates tags that are still inactive. The read role carries one write permission for this: changing the status of cost allocation tags. It can’t change anything else in your account. Activation isn’t retroactive. Spend from before a tag was activated doesn’t carry it. To fill in that history, an owner of the account can request an AWS backfill. The Activate cost allocation tags option is on by default. Clear it to opt out, and CostGraph then only reads the tags you’ve already activated. Connections created before this option existed aren’t activated until you reconnect them.

Push cost data

If you would rather not grant a cross-account role, run the exporter in your own account instead. It reads the same Data Export from S3 and pushes the records, so nothing but the FOCUS rows leaves your account:

provider: objectstore
sink: costgraph
OBJECTSTORE_BUCKET: my-focus-export
OBJECTSTORE_PREFIX: focus-1-2/
OBJECTSTORE_LAYOUT: manifest

No keys: give the exporter’s role s3:ListBucket and s3:GetObject on the export bucket and it uses the role it runs as. manifest reads exactly the files the export names, which is what AWS writes beside the data.

Cost processing

Cost: FOCUS line items normalized to the same canonical model as every other provider, keeping billed and effective cost, credits, and commitment attribution.
Reconciliation: allocated plus unallocated cost is checked to equal the invoice per billing account per month.
Allocation: spend attributed by resource, tag, and account, with an explicit unallocated bucket for tax, shared, and commitment costs.

Network access

Network access is the quickest way to fill in Context when only your bill is connected. With it, CostGraph reads your AWS inventory and connects what you run. Context shows each VPC and subnet with its instances and EKS clusters, and the volumes, file systems, NAT gateways, and load balancers they use. When network flow tracing also runs, Context names the private traffic between them. Network access is read-only and on by default. It covers the account where you create the stack or run the script. It doesn’t read data, logs, or secrets. To also see workloads, usage, and live traffic, install the CostGraph Agent or the CostGraph Operator.

Enable network inventory for an existing connection

If you set up the account with the stack, select Grant network access in Context. It opens the update for your CostGraph stack in us-east-1, with the current template and EnableNetworkInventory set to true. The update replaces the template, because a stack created before network access existed has no EnableNetworkInventory parameter. Review it, acknowledge that the stack changes IAM resources, and select Submit. When the update finishes, the NetworkInventoryStatus output reads enabled. If you set up the account with the CloudShell script, rerun it with COSTGRAPH_ENABLE_NETWORK_INVENTORY=1:

curl -sSL https://setup.costgraph.ai/integrations/aws.sh \
  | COSTGRAPH_EXTERNAL_ID="org_..." COSTGRAPH_ENABLE_NETWORK_INVENTORY=1 sh

If you manage the role yourself, add an inline policy named CostGraph-NetworkRead with the actions listed in Read access. You don’t need to reconnect. CostGraph picks up the new access within the hour.

Savings recommendations

With recommendations on, AWS’s own savings recommendations appear in Insights next to CostGraph’s. Each one shows two numbers: AWS’s estimate, and CostGraph’s price calculated from your bill.

Purchase recommendations (Savings Plans and Reserved Instances) and recommendations CostGraph can’t price yet show the monthly spend they affect, not a saving. They don’t add to the savings total.
When CostGraph already flags the same resource, you see one item that says whether AWS agrees.

The connection card in Integrations tells you what to do if access or enrollment is missing.

Turn them on for an existing connection

Update the CostGraph stack in AWS and turn recommendations on. CostGraph picks up the change automatically. If you set up the account with the CloudShell script, rerun it instead. If you also cleared Show AWS savings recommendations when you connected, connect the account again with the option on. If you manage the role yourself, run the two commands from Setup with Terraform from the management account, one time. Also add the CostGraph-RecommendationsRead policy to the read role, as shown there. Recommendations appear up to 24 hours after you turn them on. A connection to a member account only shows recommendations if the management account turned them on for member accounts.