Skip to content

Wire the Athena CUR datasource so the finance board's cost-reconciliation panels work in prod #59

Description

@stxkxs

Problem

The agent-finance dashboard's four reconciliation panels set "datasource": "athena-cur"
(dashboards/base/platform/agent-finance.yaml:50,66,143,169), but no Athena GrafanaDatasource
CR exists and the AMG workspace neither enables the Athena plugin nor grants the role Athena/Glue/S3
access. So the board's reason to exist — CUR-truth spend vs the operator's in-cluster estimate, and
estimate-drift — is dead in prod. The CloudWatch panels were fixed in #58; this is the remaining
datasource gap on that board.

The Athena backend is real: the eks-agent-platform cost-pipeline provisions the Athena workgroup,
Glue database, CUR tables, and the invocation_cost_reconciliation view.

Fix (cross-repo)

landing-zone (managed-monitoring component):

  • Add ATHENA to aws_grafana_workspace.data_sources (currently [PROMETHEUS, CLOUDWATCH]) so AMG
    enables the grafana-athena-datasource plugin.
  • Grant the AMG service role athena:StartQueryExecution/GetQueryExecution/GetQueryResults,
    glue:GetTable/GetDatabase/GetPartitions, and s3:GetObject/ListBucket on the CUR results bucket +
    the Athena query-output location.

eks-gitops (dashboards/base/datasources/):

  • Add athena-cur.yaml GrafanaDatasource (type grafana-athena-datasource, uid athena-cur,
    jsonData: catalog AwsDataCatalog, database, workgroup, defaultRegion us-west-2, output-location).
  • The database/workgroup/output-location are per-env (each env's cost-pipeline) — wire them as a
    PLACEHOLDER + per-env overlay patch (same pattern as the AMP workspace URL), sourced from the
    cost-pipeline SSM/Terraform outputs.
  • The finance board references a ${cost_database} it doesn't define yet — add the templating var.

Why not done inline

Cross-repo (landing-zone IAM + plugin + eks-gitops CR + per-env workgroup wiring), and the Athena
plugin/IAM path can't be validated offline — it needs a live AMG workspace to confirm the query path.
Until then the four finance reconciliation panels stay no-data; the board's CloudWatch + kube_customresource
spend panels work.

Acceptance

  • landing-zone AMG enables Athena + role can query the CUR view.
  • eks-gitops athena-cur datasource CR renders in all overlays with per-env workgroup substitution.
  • The four finance panels return CUR data against a live workspace.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions