Skip to content

validate/doctor: add Dashboard widget reference integrity check (dataset → dimensions/measures + chartConfig fields) #1721

Closed

Description

@os-zhuang

Problem

The v9 single-form cutover (ADR-0021) makes every DashboardWidget bind a semantic dataset and select its dimensions/values (measures) by name (e.g. sum_amount, ticket_count). But nothing validates that those names — or the chartConfig axis fields that mirror them — actually resolve against the referenced dataset. A widget can name a dataset that doesn't exist, a measure the dataset never declares, or point chartConfig.yAxis[].field at a stale base column, and every build-time tool still passes:

  • objectstack validate — passes. It only runs ObjectStackDefinitionSchema.safeParse(...) (packages/cli/src/commands/validate.ts). In the schema, chartConfig.yAxis[].field / xAxis.field / series[].name are plain z.string() (packages/spec/src/ui/chart.zod.ts:81,108, "Data field key"), and a widget's dataset / dimensions / values are just identifier strings with no cross-reference to the dataset's declared dimensions[] / measures[] (packages/spec/src/ui/dashboard.zod.ts:156-160). Shape is valid; the reference is never checked.
  • objectstack lint — no relevant rule. The only data-model rules are the relationship conventions in packages/cli/src/lint/data-model-rules.ts (e.g. relationship/line-item-should-be-master-detail) plus i18n coverage. Nothing for dashboards/datasets/charts.
  • objectstack doctor — already does cross-reference integrity for views (findOrphanViews → "View integrity — All views reference valid objects", packages/cli/src/commands/doctor.ts:72), unused objects, and circular deps — but has no equivalent for dashboard widgets.
  • objectstack build (= compile) — same schema/protocol checks as validate; doesn't catch it.

The result is a silent failure class: build is green, the page is broken.

Impact (downstream evidence)

The cutover broke two widget classes in objectstack-ai/templates with a fully green build:

  1. Record-listing type:'table' widgets bound to aggregate datasets with only a count measure and no dimension collapsed to a single number (they were meant to list rows). Fixed manually by re-modeling as object-bound ListViews — templates#27 (19 widgets across 11 dashboards).
  2. Chart widgets (bar/line/pie) whose chartConfig.yAxis[].field still named the old base column (amount, total_amount, sometimes a placeholder value) while post-cutover data is keyed by the dataset's measure name (sum_amount, ticket_count). Field doesn't resolve → axis labels render but the series is empty. A second variant had no chartConfig at all, so the component can't tell which measure to plot → also empty. 17 charts across the 11 dashboards rendered blank. Fix incoming (branch fix/dashboard-chart-bindings).

Both were invisible until someone opened each page. A build-time check would have caught all 36 at author time.

Minimal reproduction

A dataset declaring measure sum_amount, and a bar widget whose yAxis.field names the old base column amount:

// dataset: measure is "sum_amount"
const expenseLineMetrics: Dataset = {
  name: 'expense_line_metrics',
  object: 'expense_line',
  dimensions: [{ name: 'category', field: 'category' }],
  measures: [{ name: 'sum_amount', aggregate: 'sum', field: 'amount', certified: true }],
};

// widget: values is correct, but chartConfig.yAxis.field points at the raw column
const widget = {
  id: 'spend_by_category',
  dataset: 'expense_line_metrics',
  dimensions: ['category'],
  values: ['sum_amount'],
  type: 'bar',
  chartConfig: {
    type: 'bar',
    xAxis: { field: 'category' },
    yAxis: [{ field: 'amount' }], // ❌ no measure named "amount"; data returns "sum_amount"
  },
};

objectstack validate and objectstack build both pass. At runtime the bar chart shows the category axis and an empty series. The same happens if chartConfig is omitted on a chart-type widget.

Expected

A build-time check (no running server needed) should fail or warn with a specific, actionable message, e.g.:

✖ Dashboard widget integrity
  Widget "spend_by_category": chartConfig.yAxis[0].field "amount" does not
  resolve to a measure of dataset "expense_line_metrics" (declared measures:
  sum_amount). Did you mean "sum_amount"?

  Widget "spend_by_category": chart-type widget has no chartConfig — the
  component cannot determine which measure to plot.

⚠ Widget "pending_reports_table": type 'table' is bound to dataset
  "expense_report_metrics", which declares only a count measure and no
  dimension. This is almost certainly a flat record listing — model it as an
  object-bound ListView (ADR-0017), not a dashboard widget.

Proposed: "Dashboard widget reference integrity" check

Add a check (mirroring the existing findOrphanViews view-integrity pass in doctor.ts) that resolves each DashboardWidget against its dataset. For every widget assert:

  • (a) dataset resolves to a declared Dataset.
  • (b) every dimensions[] entry is a real dimension name on that dataset (DatasetDimensionSchema.name).
  • (c) every values[] entry is a real measure name on that dataset (DatasetMeasureSchema.name).
  • (d) if chartConfig is present: xAxis.field resolves to one of the widget's dimensions (or a dataset dimension); each yAxis[].field and series[].name resolves to one of the widget's measures (values). For a chart-type widget (bar/line/pie/…) with no chartConfig, warn that no measure mapping exists.
  • (e) warn when a type:'table'/'pivot' widget is bound to a dataset that declares only a count-style measure and no dimension — almost certainly a flat record listing that should be a ListView (ADR-0017).

Best landing spot: the same place views are already checked. Two complementary options:

  1. doctor — extend the integrity section (findOrphanViews/findUnusedObjects, packages/cli/src/commands/doctor.ts:72-156) with findOrphanWidgets(config) that walks dashboards[].widgets[] and resolves dataset → dimensions/measures the same way. Lowest-friction; reuses the existing reporting UX ("View integrity … / Dashboard integrity …").
  2. validate — add a semantic pass after safeParse succeeds (packages/cli/src/commands/validate.ts), so the failure surfaces in CI/build where the cutover damage actually occurred. (a)–(c) are hard errors; (d) error on a present-but-wrong field, warn on missing chartConfig; (e) is a warning.

There's a clean precedent for the name-resolution itself: the dataset schema's superRefine already resolves derived measure references by name within a dataset (packages/spec/src/ui/dataset.zod.ts, e.g. "derived measure cannot reference itself" / references-undeclared-measure). This proposal is the same resolution, extended across the dashboard → dataset → dimension/measure boundary. Doing it in validate/doctor (rather than the spec schema) keeps it where multi-package config is already assembled, so a widget in one package can resolve a dataset in another.

References

  • objectstack-ai/templates #27 — record-listing tables → ListViews (downstream fix for failure class 1).
  • Forthcoming objectstack-ai/templates PR — "fix 17 chart widget bindings" (failure class 2).
  • ADR-0021 (docs/adr/0021-analytics-dataset-semantic-layer.md) — single-form widget cutover.
  • ADR-0017 (docs/adr/0017-object-has-many-view.md) — object-bound ListViews for record listings.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions