Analytics datasets: duty health, stagnation, workload

src/datasets/duty-health.dataset.ts

@@ -0,0 +1,128 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import { defineDataset } from '@objectstack/spec/ui';
+
+import { governed } from './governed.js';
+
+/**
+ * `duly_duty_health` — the on-time picture, over GOVERNED work only.
+ *
+ * Dashboards bind datasets, never objects (ADR-0021): a widget that names a
+ * dimension or measure this file does not declare renders an empty chart and
+ * reports success, so the names below are a contract. `#10` binds them.
+ *
+ * ── What is NOT here, and why it is upstream rather than worked around ────
+ *
+ * The card asks for three more measures — `done on time`, `late`, and the
+ * `on-time rate` derived from them. All three reduce to ONE comparison:
+ *
+ *     completed_at <= due_date + duty.grace_days
+ *
+ * That comparison cannot be expressed by a dataset measure filter, and the
+ * reason is worth stating precisely, because one third of it DOES work and
+ * the other two thirds are separate platform gaps:
+ *
+ * 1. **Reaching the related field works.** `include: ['duty']` plus a
+ *    `duty.grace_days` path is exactly what the semantic layer is for — joins
+ *    are compiled from `include` (ADR-0071, ≤3 hops) and the author writes no
+ *    ON clause. `frequency` below is that same reach, in production, so this
+ *    half is proven rather than assumed.
+ *
+ * 2. **Comparing two COLUMNS is refused on the SQL path.** The filter grammar
+ *    declares a field reference — `{ completed_at: { $lte: { $field:
+ *    'due_date' } } }` — and `@objectstack/spec`'s own `filter.zod.ts` records
+ *    that `driver-sql` (and `driver-sqlite-wasm`, which inherits its compiler)
+ *    reject it with `INVALID_FILTER` / HTTP 400, while the in-memory evaluator
+ *    resolves it. Tracked upstream as objectstack#5222. That split is worse
+ *    than a uniform gap for a DATASET in particular: the same declaration
+ *    would answer on a memory driver and 400 on a SQL one, so the measure's
+ *    correctness would depend on the deployment.
+ *
+ * 3. **There is no date arithmetic in the grammar at all.** `FILTER_OPERATORS`
+ *    is closed — equality, ordering, set, range, string, null/exists — and
+ *    nothing adds an interval to a column. The `{N_days_ago}` macro
+ *    vocabulary (`DATE_MACRO_PARAM_RE`) is relative to NOW, never to another
+ *    column, so it cannot express "+ grace_days" either. And here the interval
+ *    is itself a column, which is strictly harder than a literal offset. So
+ *    even if #5222 landed tomorrow, `due_date + duty.grace_days` would still
+ *    have no spelling.
+ *
+ * The two workarounds were both considered and both rejected on the card's own
+ * terms. Denormalising `grace_days` (or a pre-computed `grace_deadline`) onto
+ * `duly_task` is a second writer that drifts the day a duty's grace is edited —
+ * `AGENTS.md` rule 5 forbids it outright. Reducing the rate in TypeScript over
+ * query results is the hand-written aggregation the metadata-first instruction
+ * on this card exists to prevent, and it would also put the number outside the
+ * semantic layer where no dashboard could bind it.
+ *
+ * So the measures are absent rather than approximated. An `on_time_rate` that
+ * silently ignored grace would be wrong in the direction that matters — it
+ * would mark late every task completed inside the grace its own duty grants —
+ * and it would be wrong invisibly, which is how a number nobody trusts becomes
+ * the number everybody reports. Note what the gap currently costs: `grace_days`
+ * is authored on `duly_catalog_item`, propagated to `duly_duty` at
+ * instantiation, and read by NOTHING. This dataset was its only intended
+ * consumer.
+ *
+ * Filed upstream as **objectstack-ai/objectstack#14104**, with parts 2 and 3 above
+ * as the two independent halves. Do not close this hole locally: an approximation
+ * here is exactly how a platform gap becomes permanent and invisible.
+ *
+ * ── `tasks_due` excludes cancelled, in every dataset that uses the name ───
+ * A cancelled task was withdrawn: it was never owed, so it is neither load nor
+ * the denominator of anything. Keeping it out is also what makes the counts
+ * add up — due = done + skipped + still-open — which a dashboard author will
+ * assume whether or not anyone tells them. `duly_workload.tasks_due` carries
+ * the identical definition on purpose: one name, one meaning, across the
+ * semantic layer.
+ */
+export const DutyHealth = defineDataset({
+  name: 'duly_duty_health',
+  label: 'Duty health',
+  description:
+    'On-time picture over governed duties (role catalog and manager-assigned). Self-declared work is surfaced in the task views and never scored here.',
+
+  object: 'duly_task',
+
+  // `duty` is joined for `frequency` only — the task itself carries owner,
+  // business unit, period and caliber, denormalised at dispatch so a rollup
+  // survives a later transfer or a re-sourced duty.
+  include: ['duty'],
+
+  dimensions: [
+    { name: 'business_unit', label: 'Business unit', field: 'business_unit', type: 'lookup' },
+    { name: 'owner', label: 'Owner', field: 'owner', type: 'lookup' },
+    { name: 'period_key', label: 'Period', field: 'period_key', type: 'string' },
+    { name: 'frequency', label: 'Frequency', field: 'duty.frequency', type: 'string' },
+    // Available so the governed population can be split by where the work came
+    // from. Slicing by this NEVER reveals self-declared work — every measure
+    // is filtered to catalog+assigned, so the `self` bucket is empty by
+    // construction. That is the design, not an oversight: see `governed.ts`.
+    { name: 'source', label: 'Source', field: 'source', type: 'string' },
+  ],
+
+  measures: [
+    {
+      name: 'tasks_due',
+      label: 'Tasks due',
+      aggregate: 'count',
+      filter: governed({ status: { $ne: 'cancelled' } }),
+    },
+    {
+      name: 'tasks_done',
+      label: 'Tasks done',
+      aggregate: 'count',
+      filter: governed({ status: 'done' }),
+    },
+    {
+      // Kept separate from `tasks_done` because it is a legitimate outcome with
+      // a different meaning — "the plant was down, there was nothing to
+      // return". Folding skips into done inflates the picture; folding them
+      // into late punishes an honest answer.
+      name: 'tasks_skipped',
+      label: 'Tasks skipped',
+      aggregate: 'count',
+      filter: governed({ status: 'skipped' }),
+    },
+  ],
+});

src/datasets/governed.ts

@@ -0,0 +1,48 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import type { FilterCondition } from '@objectstack/spec/data';
+
+/**
+ * The caliber gate, in one place, for every measure in every duly dataset.
+ *
+ * ── Why this is a constant and not a convention ───────────────────────────
+ * `catalog` and `assigned` duties are what the ORGANISATION put on someone:
+ * a rate over them means something, because somebody else decided the work
+ * was owed. `self` duties are the owner's own record-keeping.
+ *
+ * A single on-time rate that quietly folds self-declared work in punishes
+ * exactly the people who declared the most — declare five extra duties, miss
+ * one, and your number goes down while the colleague who declared nothing
+ * stays at 100%. That is not a reporting inaccuracy; it is a mechanism that
+ * teaches an organisation to declare nothing, and it takes about one quarter
+ * to work. So the filter is not a default that a measure may opt out of: it
+ * is on EVERY measure in `dulyDatasets`, with no exception list, and
+ * `test/datasets.test.ts` walks the barrel to keep it that way.
+ *
+ * There is deliberately no `ungoverned()` counterpart. An exception list is
+ * the erosion path — the next reasonable-looking ticket adds one measure to
+ * it, and the one after that adds two. Self-declared work is SURFACED, in the
+ * operational views (`src/views/task.view.ts` carries `source` as a column and
+ * scores nothing); it is not surfaced through the metric layer.
+ *
+ * `source` remains a DIMENSION on the datasets so the governed population can
+ * still be split by where the work came from — "how much of this unit's load
+ * is manager-assigned rather than role-catalog?" is a real question, and it is
+ * answerable without ever putting `self` into a score.
+ */
+export const GOVERNED_SOURCES = ['catalog', 'assigned'] as const;
+
+/**
+ * `source IN ('catalog','assigned')`, ANDed with whatever else the measure
+ * needs. A `FilterCondition` is a record of field conditions combined with
+ * AND, so spreading is the whole implementation — no `$and` wrapper needed.
+ *
+ * Read `duly_task.source` and not `duty.source`: the task carries its own copy,
+ * stamped at dispatch, so a task keeps the caliber it was dispatched under even
+ * if the duty is later re-sourced. That is history, not drift — and it means a
+ * governed measure does not depend on the `duty` join being present.
+ */
+export const governed = (extra: FilterCondition = {}): FilterCondition => ({
+  source: { $in: [...GOVERNED_SOURCES] },
+  ...extra,
+});

src/datasets/index.ts

@@ -12,5 +12,27 @@
 // resolves it against the keyed branch of `MetadataCollectionInput`, which
 // makes `name` optional and fails the assignment. A named array is `never[]`
 // while empty and infers correctly the moment something is pushed into it.
+//
+// ⚠ A dataset is a CONTRACT, not an implementation detail. Dashboards and
+// reports bind dimensions and measures BY NAME (ADR-0021), and a widget naming
+// one that does not exist renders an empty chart and reports success. Renaming
+// or dropping anything declared in these files breaks its consumers silently —
+// grep the dashboards barrel before you touch a name.
+//
+// ⚠ And the binding UNDER a dataset is not checked either: `objectstack validate`
+// and `objectstack build` both exit 0 on a dataset whose base `object`, `include`
+// path or dimension/measure `field` paths name nothing at all (measured, and filed
+// as objectstack-ai/objectstack#14105 — a bad date-macro TOKEN in the same measure
+// filter IS caught, so the traversal exists and only the reference resolution is
+// missing). Until that lands, `test/datasets.test.ts` pinning the field paths is
+// the only thing standing between a typo here and a chart that renders empty while
+// every gate reports success.
+
+import { DutyHealth } from './duty-health.dataset.js';
+import { Stagnation } from './stagnation.dataset.js';
+import { Workload } from './workload.dataset.js';
+
+export { DutyHealth, Stagnation, Workload };
+export { GOVERNED_SOURCES, governed } from './governed.js';
 
-export const dulyDatasets = [];
+export const dulyDatasets = [DutyHealth, Stagnation, Workload];

src/datasets/stagnation.dataset.ts

@@ -0,0 +1,112 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import { defineDataset } from '@objectstack/spec/ui';
+
+import { governed } from './governed.js';
+
+/**
+ * `duly_stagnation` — the early-warning picture, and the reason this product
+ * does not carry a completion percentage.
+ *
+ * ── Stagnation is NOT lateness ────────────────────────────────────────────
+ * Nothing below mentions `due_date`. That is the entire point and it is the
+ * one thing a reviewer should check first: a task that is untouched for three
+ * weeks and not due for another two is ALREADY the signal, and a lateness
+ * measure cannot see it until it is too late to act. Lateness tells a manager
+ * about work that has already failed; stagnation tells them about work that is
+ * quietly going nowhere, weeks earlier, while intervening is still cheap.
+ *
+ * A percentage would answer neither. It is a number nobody can verify, which
+ * is exactly why it becomes the number everyone reports — and 80% on an
+ * untouched task is 80% forever. `last_update_at` cannot be talked up: it is
+ * server-owned, stamped by `src/hooks/task.hook.ts` only when `status`, `note`
+ * or `skip_reason` actually CHANGED, never on an administrative or bulk write.
+ * Re-owning, re-dating or importing a task does not reset the clock.
+ *
+ * ── Why the buckets are expressible, when the on-time rate is not ─────────
+ * These thresholds are relative to NOW, not to another column, so they are
+ * exactly what the date-macro vocabulary is for. `{7_days_ago}` and friends
+ * are resolved server-side by `resolveFilterTokens`, which
+ * `@objectstack/spec`'s `date-macros.zod.ts` names as wired into "the
+ * analytics dataset executor" specifically — dataset definitions being one of
+ * the filter sources that never pass through a renderer. Unknown tokens fail
+ * the build rather than comparing as a literal string and matching nothing,
+ * which is why the spellings below are safe to trust; `test/datasets.test.ts`
+ * pins them against the spec's own token grammar.
+ *
+ * Contrast `duly_duty_health`, where the comparison is column-to-column plus
+ * arithmetic and there is no such vocabulary. The difference between the two
+ * cases is the whole answer to "can a dataset say this".
+ *
+ * ── Shape a dashboard author must know: the buckets are CUMULATIVE ────────
+ * `untouched_over_14d` counts everything `untouched_over_30d` counts. They are
+ * nested thresholds, not disjoint bands, so they must not be summed and they
+ * must not go in a pie chart. Stack them as thresholds, or difference them in
+ * the widget if disjoint bands are wanted. Named the way the card names them
+ * (">7d", ">14d", ">30d") precisely so the nesting is legible from the name.
+ *
+ * `last_update_at` is never null on a task that reached the database — the
+ * lifecycle hook stamps it on `beforeInsert` ("a brand-new task has just been
+ * touched, by definition"), so a freshly dispatched task cannot slip out of
+ * these counts through a NULL comparison.
+ */
+export const Stagnation = defineDataset({
+  name: 'duly_stagnation',
+  label: 'Stagnation',
+  description:
+    'Open governed work that is not moving, bucketed by how long it has gone untouched. Deliberately independent of due date — an untouched task that is not yet due still stagnates.',
+
+  object: 'duly_task',
+
+  dimensions: [
+    { name: 'business_unit', label: 'Business unit', field: 'business_unit', type: 'lookup' },
+    { name: 'owner', label: 'Owner', field: 'owner', type: 'lookup' },
+  ],
+
+  measures: [
+    {
+      name: 'open_tasks',
+      label: 'Open tasks',
+      aggregate: 'count',
+      filter: governed({ status: { $in: ['open', 'in_progress'] } }),
+    },
+    {
+      name: 'untouched_over_7d',
+      label: 'Untouched > 7 days',
+      aggregate: 'count',
+      filter: governed({
+        status: { $in: ['open', 'in_progress'] },
+        last_update_at: { $lt: '{7_days_ago}' },
+      }),
+    },
+    {
+      name: 'untouched_over_14d',
+      label: 'Untouched > 14 days',
+      aggregate: 'count',
+      filter: governed({
+        status: { $in: ['open', 'in_progress'] },
+        last_update_at: { $lt: '{14_days_ago}' },
+      }),
+    },
+    {
+      name: 'untouched_over_30d',
+      label: 'Untouched > 30 days',
+      aggregate: 'count',
+      filter: governed({
+        status: { $in: ['open', 'in_progress'] },
+        last_update_at: { $lt: '{30_days_ago}' },
+      }),
+    },
+    {
+      // The single oldest untouched moment in the group. A KPI tile bound to
+      // this answers "what is the worst thing here" without ranking anybody
+      // against anybody — it is a timestamp, not a score, and it names a date
+      // rather than a person.
+      name: 'oldest_last_update_at',
+      label: 'Oldest touch',
+      aggregate: 'min',
+      field: 'last_update_at',
+      filter: governed({ status: { $in: ['open', 'in_progress'] } }),
+    },
+  ],
+});

src/datasets/workload.dataset.ts

@@ -0,0 +1,57 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import { defineDataset } from '@objectstack/spec/ui';
+
+import { governed } from './governed.js';
+
+/**
+ * `duly_workload` — the forward look: which period is about to overload.
+ *
+ * The only dataset here that is about work not yet done. Its job is to make a
+ * pile-up visible while it can still be moved — a quarter-end where four
+ * annual duties, a semi-annual and the usual monthlies all land in the same
+ * week is a problem you can only solve in advance.
+ *
+ * ── Shape a dashboard author must know ────────────────────────────────────
+ * `due_week` and `due_month` are the SAME column at two granularities, not two
+ * columns. Group by one or the other, never both at once — crossing them
+ * produces one populated cell per week and empties everywhere else. Weeks are
+ * ISO (Monday start) wherever the driver does not truncate server-side.
+ *
+ * There is no status dimension and no per-person volume measure. "Who has the
+ * most tasks" is not a question this dataset answers, and that is deliberate:
+ * `owner` is here so a manager can see whether ONE person's next fortnight is
+ * unsurvivable, which is a workload question, not a ranking. A measure built
+ * to compare item counts BETWEEN people is the specific mechanism by which the
+ * caliber separation gets undone one reasonable-looking ticket at a time, so
+ * this dataset carries exactly one measure and it is the same one
+ * `duly_duty_health` carries.
+ *
+ * `tasks_due` is identical to `duly_duty_health.tasks_due` — governed sources,
+ * cancelled excluded. One name, one meaning, across the semantic layer; a
+ * cancelled task was withdrawn, so it is not future load.
+ */
+export const Workload = defineDataset({
+  name: 'duly_workload',
+  label: 'Workload',
+  description:
+    'Governed tasks due, bucketed forward by week and by month, so a period that is about to overload is visible while it can still be rebalanced.',
+
+  object: 'duly_task',
+
+  dimensions: [
+    { name: 'business_unit', label: 'Business unit', field: 'business_unit', type: 'lookup' },
+    { name: 'owner', label: 'Owner', field: 'owner', type: 'lookup' },
+    { name: 'due_week', label: 'Due (week)', field: 'due_date', type: 'date', dateGranularity: 'week' },
+    { name: 'due_month', label: 'Due (month)', field: 'due_date', type: 'date', dateGranularity: 'month' },
+  ],
+
+  measures: [
+    {
+      name: 'tasks_due',
+      label: 'Tasks due',
+      aggregate: 'count',
+      filter: governed({ status: { $ne: 'cancelled' } }),
+    },
+  ],
+});