Skip to content

Analytics summary CSV concatenates three heterogeneous sections with comment lines: no RFC 4180-compliant output exists #367

Description

@usmanimamu17-create

Problem

export_analytics_summary (app/utils/analytics_exporter.py) builds a single CSV by concatenating three differently-shaped sections:

buffer.write("# KPI Metrics\n")
kpi_writer = csv.DictWriter(buffer, fieldnames=summary["kpi"].keys())
...
buffer.write("# Trends Data\n")
trend_writer = csv.DictWriter(buffer, fieldnames=trend_data[0].keys())

Consequences:

  • No standard parser can consume the file: RFC 4180 CSVs have one header row and uniform columns; this file has #-prefixed section headers, blank separator lines, and three different column sets, so csv.reader, Excel, and analytics importers either fail or misalign rows.
  • The JSON form of the same function is clean and structured ({"kpi": ..., "trends": [...], "trend_count": N}), so the CSV path provides a strictly worse contract for the same data.
  • Empty trends fall back to a hardcoded header ("date,total_outages,violations,rewards,penalties") that only matches SLATrendPoint by luck; any future field addition silently produces a malformed file.

Root cause

The CSV branch was written as "dump each section" rather than a defined CSV schema.

Why this is architecturally hard

  1. A compliant CSV needs a single unified schema (e.g. one row per trend point with KPI values repeated, or a documented multi-table layout like table:column prefixing) — a format decision that consumers (dashboard exports, BI tooling) must agree on.
  2. export_dashboard_kpi and export_trends each already produce valid single-table CSVs; the summary function could reuse them or return JSON only, which is a product decision about what "export" means.
  3. The OpenAPI snapshot documents format=csv as an option; changing the output shape or removing CSV is a contract change for the frontend export UI.

Proposed design

Either produce one RFC 4180-compliant CSV with a unified schema (documented), or make format=csv return the KPI CSV (and document trends/aggregation as JSON-only); add tests that parse the CSV with csv.reader and assert uniform columns.

Acceptance criteria

Service

  • CSV output parses with a standard CSV reader into uniform rows.
  • Empty datasets produce the same schema as non-empty ones.

Tests

  • A parser test asserts uniform columns for populated and empty data.
  • Existing analytics export tests pass.

Out of scope

The trend-empty hardcoded header mismatch (resolved by the same fix) and streaming.

Getting started

pytest tests/test_sla_typing.py -q
make typecheck

Good first files to read: app/utils/analytics_exporter.py, app/api/v1/endpoints/sla.py.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardStellar WaveIssues in the Stellar wave programThird CampaignCampaign: Third Campaignarea/apiREST API surface, OpenAPI, pagination, content-negotiation, error responsesarea/dataImported campaign issue labelpriority/mediumStandard backlog item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions