Skip to content

experiments/interface-ab: generate the governed tier from the derived schema (DSL) — stacked on #26 - #29

Open
shyan-ai wants to merge 2 commits into
devrev:mainfrom
shyan-ai:community/dsl-generated-governed-tier
Open

experiments/interface-ab: generate the governed tier from the derived schema (DSL) — stacked on #26#29
shyan-ai wants to merge 2 commits into
devrev:mainfrom
shyan-ai:community/dsl-generated-governed-tier

Conversation

@shyan-ai

Copy link
Copy Markdown

Stacked on #26 (the interface-ab context-tier pack). This branch is #26 + one commit
(1c89a02) that makes the analytic governed views generated from the derived schema by a small DSL
instead of hand-written
. Because #26 isn't merged yet, the diff below includes its content — the
new work is that single commit
; review it directly, or fold this into #26, whichever you prefer.

What the new commit does (experiments/interface-ab/ops-context/)

The analytic governed views in server.py are now generated, not hand-coded:

  • ops_arr_at_risk_by_arearevenue_at_risk_by_grain over the tagged part, additively rolled up
    to product_area (finest grain retained; accounts de-duped per area).
  • ops_open_p1_sla_statusthreshold_breach classified by account tier. The join and the age
    math are generated; the SLA thresholds and MSA docs stay the binding Tier-1.5 config.
  • ops_linkage_map_derived → a real scanner (primary-key inference + value-set-overlap FK
    detection + self-referential hierarchy + junction detection). The previous "MECHANICALLY DERIVED /
    no human input" map was in fact hand-authored (5 hardcoded reference rows); the scanner genuinely
    derives the topology (13 references) from the data. This is a small correctness/honesty fix that
    stands on its own.
  • Auxiliary, non-analytic tools stay hand-coded (ops_account_360, ops_msa_sla_terms, the
    declarative ops_linkage_map, ops_skill_sla_analysis, ops_search_accounts).

Why

A governed view is a join path + a grouping grain + an aggregate + a filter + an exclusion rule, with
provenance — a small, closed set of operations (relational algebra plus a few analytic operators). So
a governed context tier need not be per-view hand-coding: implement the operators once, verify them
once, and express each view as a short spec over an auto-derived linkage graph.

Correctness

Answer-equivalence to the prior hand-written views is proven in ops-context/dsl/verify.py
(run: DATA_DIR=/path/to/data python verify.py):

  • arr_at_risk_by_area → $912K / $502K / $472K / $280K across 4 product areas
  • open_p1_sla_status → 27 breaches / 2 not-breached / 2 excluded (Starter, no SLA)
  • linkage_map_derived → the real join graph, genuinely derived

Same answers as before, so the A/B findings in #26 hold unchanged.

Provenance / dependencies

The engine (dsl/scan.py, dsl/dsl.py) is vendored unmodified (bar a one-line header) from
floormap — a standalone, Apache-2.0, dependency-free
governed analytic engine; dsl/LICENSE included. No new runtime dependencies (stdlib only); the
Dockerfile copies dsl/. Canary strings preserved.

shyan-ai and others added 2 commits August 5, 2026 23:15
…xt server

Follow-up to PR devrev#10 — the second half of the contribution package,
raised to make the placement discussion concrete. Proposes experiments/
as a home for community reproductions. Canary embedded in every file;
make validate passes; no data or task changes.
… schema (DSL) instead of hand-writing it

The analytic governed views in ops-context/server.py are now GENERATED by a small vendored DSL over
an auto-derived linkage graph, not hand-coded:
- ops_arr_at_risk_by_area -> revenue_at_risk_by_grain (+ additive product_area roll-up)
- ops_open_p1_sla_status  -> threshold_breach classified by account tier (SLA thresholds = binding Tier-1.5 config)
- ops_linkage_map_derived -> the real scanner (PK inference + value-overlap FK detection + self-ref +
  junctions), replacing the former hand-authored map (the honest 'no human input')

Answer-equivalence to the prior hand-written views is proven in ops-context/dsl/verify.py
(912K/502K/472K/280K across 4 areas; 27/2/2 SLA; the real join graph). The engine (scan.py, dsl.py) is
vendored unmodified from github.com/JieGouAI/floormap (Apache-2.0, dependency-free). Auxiliary
non-analytic tools stay hand-coded; Dockerfile copies dsl/; no new runtime deps.
@nimit2801

Copy link
Copy Markdown
Contributor

Thanks for the contribution. First-touch impression: this looks like an experimental interface/context-tier PR stacked on #26, with the new delta focused on generating governed analytic views from a derived schema via a vendored Apache-2.0 DSL.

Checks: I do not see any status checks reported on the PR yet, so this should be reassessed once CI or validation results are available.

Main risk: because this is stacked on #26 and touches benchmark experiment behavior plus vendored DSL code, maintainers should separate the one-commit DSL delta from the base experiment before reviewing correctness.

Suggested maintainer verification:

  1. Run the stated verifier against the expected data path and confirm answer-equivalence for ARR-at-risk, P1 SLA status, and derived linkage map.
  2. Confirm benchmark/task safety: canary preservation, synthetic-data boundaries, no answer leakage through generated views, and reproducibility via the repo validation path such as make validate if available.
  3. Check the vendored floormap provenance/license and ensure the DSL copy is intentionally scoped with no hidden runtime dependency changes.

I will reassess after checks are added or complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants