Regulations tracker with AI regulation impact analysis#4198
Draft
gorkem-bwl wants to merge 95 commits into
Draft
Regulations tracker with AI regulation impact analysis#4198gorkem-bwl wants to merge 95 commits into
gorkem-bwl wants to merge 95 commits into
Conversation
Design for a Regulations Tracker module mirroring AI Trust Index: weekly BullMQ poll of the public Global AI Regulations feed, per-country hash-based change detection, per-org tracking + configurable recipients, in-app + email notifications, and a Browse/Tracked/Settings/Detail UI. Includes source-feed evaluation and consolidated edge-case handling (feed-floor guards, soft-delete by presentSlugs, first-seed suppression, cosmetic-hash unstructured path, HTML escaping, BullMQ obliterate ordering).
15 tasks across 5 phases (migration/models/seed, feed+utils, weekly job+notifications+template, routes+controllers, frontend), mirroring the AI Trust Index module with complete code per step and TDD cycles.
## Changes - Create Servers/utils/regulationsTracker.utils.ts with renderChangeLine, escapeHtml, currentIsoWeek (pure), getMetaQuery, upsertFeedTx (transactional) - Create Servers/utils/__tests__/regulationsTracker.utils.test.ts with 5 TDD tests for the 3 pure functions; all pass - Fix pre-existing unused IManifest import in regulationsTrackerFeed.ts ## Benefits - Pure functions are fully unit-tested (TDD: red → green) - upsertFeedTx is idempotent (hash-diff gating), row-lock safe, and tracks first-seed, newly-removed slugs, and per-country change lines
…task 6)
## Changes
- listCountries(filters): catalogue query with optional region/q filters
- getCountryRow(slug): single country detail including raw data blob
- listTracked(orgId): per-org tracked list joined to catalogue
- trackCountry(orgId, slug, userId): idempotent insert via ON CONFLICT DO NOTHING
- trackCountriesBulk(orgId, slugs, userId): sequential bulk wrapper
- untrackCountry(orgId, slug): delete tracked row
- getSettings(orgId): fetch notification settings row
- upsertSettings(orgId, userIds, emails, userId): ON CONFLICT DO UPDATE
- getAffectedOrgsBySlugs(slugs): find orgs tracking any of the given slugs (weekly job input)
- resolveEmailRecipients(orgId): configured user emails + free-text emails, deduped
- resolveInAppUserIds(orgId): org Admins (JOIN roles) ∪ configured user ids, deduped
## Notes
- normalizeSlug reused from existing private function (not redeclared)
- Admin-role query uses JOIN roles r ON r.id = u.role_id WHERE r.name IN ('Admin','SuperAdmin')
— confirmed against invitation.utils.ts:72 and user.utils.ts:86, exact same schema
- All tenant queries filter by organization_id
- Build: tsc clean; tests: 5/5 passed
## Changes - Add `syncRegulationsTracker.ts`: weekly BullMQ job that fetches the regulations feed, upserts the catalog via `upsertFeedTx`, fans out in-app notifications (`createNotificationQuery`) and email digests (`sendAutomationEmail`) to affected orgs. - Export `sectionMjml` pure helper (builds MJML fragments for the email digest) and `DigestItem` interface. - Suppress first-seed notifications and week-guard duplicate runs. - Add `__tests__/syncRegulationsTracker.test.ts` covering the two `sectionMjml` branches (empty list → "", HTML-escaping of names). ## Adaptations from brief - Imported `createNotificationQuery` (real export name) instead of `createNotification` (brief alias). Signature: `createNotificationQuery(notification: ICreateNotification, organizationId: number, transaction?)`. - `type` and `entity_type` cast through `unknown` because `"regulations_tracker"` and `"regulation_country"` are not yet in the `NotificationType`/`NotificationEntityType` enums; casts are safe and the DB column is VARCHAR — enums can be extended as a follow-up. - `entity_id` omitted (ICreateNotification.entity_id is `number`; country slugs are strings — storing the slug there would be a type violation). - `sendAutomationEmail(to, subject, body, attachments?)` arity matches the brief's `(emails, subject, html, undefined)` exactly.
…ests
## Changes
- Fixed silent bug where the digest email "Changed" section showed only
country names and never what changed: `ch?.detail` was always undefined
because CountryChange has no `detail` field. Now uses `ch.lines.join(", ")`
to surface the rendered change lines.
- Removed `slugs` array from the per-org bucket (was only kept for a
`void slugs` suppressor) — now unused after the removed-branch no longer
looks up changeBySlug.
- Removed `slug` array from removed-branch push; removed items carry no
change detail (correct).
- Added three mocked unit tests for syncRegulationsTracker:
1. Week-guard skip: getMetaQuery returns current ISO week → skipped, no fetch
2. First-seed suppression: wasFirstSeed=true → orgsEmailed=0, orgsNotified=0
3. Changed-country path: verifies email sent, in-app created, and digest
detail contains the joined lines string (proving the critical fix)
## Changes - automationProducer.ts: add scheduleRegulationsTrackerSync() (Monday 06:00 UTC, no obliterate) - automationWorker.ts: add dispatch branch for regulations_tracker_sync job name - producer.ts: import and call scheduleRegulationsTrackerSync() inside addAllJobs(), placed after scheduleAiTrustIndexSync() - syncRegulationsTracker.ts: remove unused _CountryChange type alias (TS6196 fix) ## Benefits - Regulations tracker feed syncs automatically every Monday at 06:00 UTC - Worker correctly routes the job to syncRegulationsTracker() - Non-obliterating scheduler — safe to run alongside other repeatable jobs
## Changes - Add Servers/controllers/regulationsTracker.ctrl.ts with 8 handlers: getCountries, getCountryDetail, getTracked (any auth user); trackCountryCtrl, trackBulkCtrl, untrackCountryCtrl, updateSettingsCtrl (admin-gated) - Add Servers/routes/regulationsTracker.route.ts with 8 routes under /api/regulations-tracker - Register route in Servers/app.ts next to aiTrustIndex mount - Regenerate swagger.yaml and docs/api-docs/src/config/endpoints.ts (677 ops, 0 drift) ## Pattern Mirrors aiTrustIndex.ctrl.ts exactly: local isAdmin inline helper, logProcessing/logSuccess/logFailure from utils/logger/logHelper, STATUS_CODE from utils/statusCode.utils.
## Changes - Add Servers/controllers/__tests__/regulationsTracker.ctrl.test.ts - Mirror aiTrustIndex.ctrl.test.ts harness: jest.mock hoisted, mockRes helper, async handlers - Cover all 8 handlers: getCountries, getCountryDetail, getTracked, trackCountryCtrl, trackBulkCtrl, untrackCountryCtrl, getSettingsCtrl, updateSettingsCtrl ## Test cases (26 total) - getCountries: 200 response, query param forwarding - getCountryDetail: live 200, 404 unknown slug, stale fallback on feed error - getTracked: 200 response, tenant isolation (org A/B scoped by organizationId) - trackCountryCtrl: 403 Editor, 403 Auditor, 400 missing slug, 200 Admin, 200 SuperAdmin, tenant isolation (orgId passed to util) - trackBulkCtrl: 403 non-admin, 400 non-array slugs, 400 non-string slug, 200 Admin - untrackCountryCtrl: 403 non-admin, 200 idempotent (never-tracked), 200 success + slug echo - getSettingsCtrl: 200 for authenticated user - updateSettingsCtrl: 403 non-admin, 400 bad email, 400 non-integer userId, 200 Admin valid input
## Changes
- Add global beforeEach(() => jest.clearAllMocks()) so mock call counts
do not accumulate across tests
- Remove now-redundant mockClear() in the "org A cannot see org B" test
(global reset makes it unnecessary)
- getCountries 200 test: assert res.json was called with the
STATUS_CODE[200]-wrapped payload { message: "OK", data: [...] },
checking that data contains an object with slug "eu"
- Auditor 403 test: add expect(trackCountry).not.toHaveBeenCalled()
mirroring the Editor 403 test, proving the admin gate blocks writes
## Changes
- Add Clients/src/application/repository/regulationsTracker.repository.ts with 8 methods (getCountries, getCountryDetail, getTracked, trackCountry, trackBulk, untrackCountry, getSettings, updateSettings) hitting /regulations-tracker/* endpoints
- Add Clients/src/application/hooks/useRegulationsTracker.ts exporting 8 hooks (useCountries, useCountryDetail, useTracked, useTrackCountry, useUntrackCountry, useTrackBulk, useSettings, useUpdateSettings) mirroring useAiTrustIndex patterns
## Details
- Repository uses apiServices from infrastructure/api/networkServices, extracts .data from all responses (STATUS_CODE envelope)
- trackCountry sends body { slug } (confirmed against trackCountryCtrl)
- trackBulk sends body { slugs } (confirmed against trackBulkCtrl)
- updateSettings sends body { recipient_user_ids, recipient_emails } (snake_case, confirmed against updateSettingsCtrl)
- Hooks use const KEY = "regulations-tracker", keepPreviousData on read queries, mutations invalidate KEY
## Changes - Add RegulationsTrackerSidebar.context.tsx — safe-context provider with trackedCount badge, mirrors AITrustIndexSidebarContext pattern exactly - Add RegulationsTrackerSidebar.tsx — SidebarShell with Browse/Tracked/Settings menu items (Settings gated behind isAdmin), mirrors AITrustIndexSidebar - Add RegulationsTracker/index.tsx — Navigate redirect to /browse, same as ATI - Add Browse/index.tsx — country catalogue with region filter + SearchBox, per-row Track/Untrack, bulk-select + Track selected, pagination - Add Tracked/index.tsx — client-side sorted/paginated tracked countries list with Untrack action, EmptyState with tips, mirrors ATI Tracked tab - Add Settings/index.tsx — admin-gated AutoCompleteField user picker + ChipInput emails, debounced auto-save via useUpdateSettings, mirrors ATI - Add CountryDetail/index.tsx — regulations list, timeline, change history; feed disclaimer rendered VERBATIM from meta.disclaimer/scopeStatement; stale:true shows warning banner + chip - Add useTrackerAlert.tsx — shared error toast hook, mirrors useTrustIndexAlert - Add 22 Regulations Tracker strings to i18n/translations.ts for de/fr/es (including truncated-prefix keys the audit script extracts from JSX) ## Gates - typecheck: pass (0 errors) - i18n:audit:strict: pass (0 gaps, 5965/5965 covered for de/fr/es)
## Changes - CountryDetail: extract stale-banner amber hex values (#FFFBEA, #B45309, #92400E) to named module-level consts (STALE_BANNER_BG/ICON_COLOR/TEXT_COLOR) with a comment explaining no matching palette token exists (same approach as AITrustIndex/AppDetail which also hardcodes its error-banner colors) - CountryDetail: py: 8 → py: "64px" (loading spinner box) - Browse: py: 6 → py: "48px" (loading spinner box) - Settings: py: 6 → py: "48px" (loading spinner box) - Tracked: py: 6 → py: "48px" (loading spinner box) - Run prettier --write; all 7 previously-warned files now pass format-check
## Changes - uiSlice.ts: add "regulations-tracker" to AppModule union type - useActiveModule.ts: detect /regulations-tracker path, navigate to /browse on switch, whitelist in localStorage validator - routes.tsx: lazy imports for RegulationsTracker index/Browse/Tracked/Settings/CountryDetail + 5 Route registrations under /regulations-tracker (bare path uses index component redirect, :slug param matches CountryDetail useParams) - Dashboard/index.tsx: wrap provider tree with RegulationsTrackerSidebarProvider - ContextSidebar/index.tsx: import context safe hook + RegulationsTrackerSidebar component, add "regulations-tracker" switch case mirroring ai-trust-index - AppSwitcher/index.tsx: add Scale icon + "Regulations tracker" module entry (sentence case) - breadcrumbs/index.tsx: skip home breadcrumb for /regulations-tracker paths ## Benefits - All /regulations-tracker/* routes are reachable via React Router - Module auto-detects from URL and persists in localStorage - Sidebar and nav entry wire up exactly following the AI Trust Index pattern
## Changes - Added German, French, and Spanish translations for the AppSwitcher module description "Track AI regulations and compliance requirements across jurisdictions" in Clients/src/i18n/translations.ts - Applied prettier formatting to routes.tsx and Dashboard/index.tsx which were flagged by the format-check gate ## Benefits - Fixes i18n:audit:strict (was 3 gaps in de/fr/es, now 0) - Fixes format-check (all files now pass Prettier style check) - All pre-PR gates pass: typecheck, i18n:audit:strict, format-check
…il proxy shape
## Changes
- listCountries now accepts organizationId as first arg and LEFT JOINs
regulation_tracked_countries to produce (t.id IS NOT NULL) AS is_tracked
on every row, scoped per-org. Column aliases prefixed with c. to avoid
ambiguity after the join.
- getCountryRow now accepts organizationId and applies the same LEFT JOIN,
returning is_tracked alongside the data JSONB column.
- getCountries controller: passes req.organizationId! as first arg to listCountries.
- getCountryDetail controller:
- Calls getCountryRow(slug, req.organizationId!) so is_tracked is available.
- Live path: spreads live + stale:false + is_tracked:local.is_tracked.
- Stale path (was broken): was returning { country: local.data } which the
frontend couldn't read. Now spreads ...local.data at root + stale:true +
is_tracked:local.is_tracked, matching the shape CountryDetail reads at
data?.data.
- Updated regulationsTracker.ctrl.test.ts mocks and assertions for the new
signatures (orgId first arg, is_tracked in mock return values).
## Benefits
- Browse page Track/Untrack label and bulk-select filtering now work correctly
per org (is_tracked was always false before).
- CountryDetail toggle correctly reflects tracking state in both live and stale
fallback paths.
- Stale fallback no longer renders blank — both paths return the same root shape.
## Blocking correctness FIX 1 — Notification enum crash (weekly job abort): - Add NotificationType.REGULATIONS_TRACKER and NotificationEntityType.REGULATION_COUNTRY to i.notification.ts - Replace `as unknown as` casts in syncRegulationsTracker.ts with real enum members - Migration 20260626114016 extends enum_notification_type + enum_notification_entity_type so Postgres INSERT no longer throws FIX 2 — Stale fallback blank page during feed outage: - CountryDetail now shows a "Last known summary" SectionCard when stale=true and no regulations/timeline/change_history are present (manifest-summary shape). Shows regulationCount + history.lastChange entries + a clear message. The live (non-stale) empty state is unchanged. FIX 3 — Bulk-track DoS guards: - trackBulkCtrl: empty array now returns 400 "slugs must be a non-empty array" and >200 slugs returns 400 "too many slugs (max 200)" — matching aiTrustIndex ## Should-fix correctness FIX 4 — Email digest double headers: - Remove static "Changed" / "No longer in the feed" headers from regulations-tracker-digest.mjml; sectionMjml already emits them. Empty sections now render no orphan header. FIX 6 — Stale fallback false-trigger on casing mismatch: - getCountryDetail now passes local.slug (canonical/normalized) to fetchCountryDetail instead of raw req.params.slug FIX 7 — Feed validation gates on valid (filtered) count: - validateManifest now filters to valid entries first, then applies ABSOLUTE_FLOOR and 50%-drop gates against validCount — a feed with many malformed entries no longer passes the gates - Updated existing tests; added two new tests covering healthy rawCount with majority-malformed entries
FIX 5 — Region dropdown collapses after filter selection:
- Browse now fetches an unfiltered country list (useCountries({})) as a stable
source for the region dropdown. Selecting a region no longer removes other
regions from the list.
FIX 8 — escapeHtml: already consolidated in regulations module.
regulationsTracker.utils.ts exports the single definition; syncRegulationsTracker
imports it from there. No code change needed; verified and documented.
FIX 9 — Shared CountryRowCard component:
- Extract CountryRowCard.tsx + CountryRow interface (shared between Browse and
Tracked). Both pages now use the component; duplicated inline markup removed.
FIX 10 — upsertFeedTx N+1 SELECT → single prefetch:
- Replace per-country SELECT inside the loop with a single
SELECT slug, hash FROM regulation_countries WHERE slug = ANY(:slugs)
into a Map, looked up per-iteration. All existing behavior preserved.
i18n: add de/fr/es translations for "Last known summary" and
"Last recorded changes" (new stale-summary banner strings from FIX 2)
…ender from DB
The catalog previously stored only the manifest summary (slug/name/region/
hash/history), so country detail pages were blank until the live feed resolved
and blank entirely when it was slow or unreachable.
## Changes
- Regenerate the seed snapshot with full per-country detail (regulations,
timeline, scope/obligations/penalties, disclaimer/meta) from the 60
/country/<slug> endpoints; stored as data = { ...country, meta }.
- upsertFeedTx accepts an optional detailBySlug map and stores full detail when
provided, falling back to the manifest summary otherwise.
- Weekly sync fetches full detail for new/hash-changed countries (via
getStoredHashes) before upserting; per-country fetch failure is non-fatal.
- Detail controller normalizes both the live and stored paths to one flat shape
(regulations/timeline/meta at the root) that the detail page reads, fixing a
live/stale shape divergence.
## Benefits
- Country detail renders complete content from our DB instantly, offline-safe.
- Fresh installs get full data day-one from the committed snapshot.
## Changes - Show the country flag emoji (from the feed) in Browse/Tracked row cards and the country detail header; globe icon remains the fallback. - Surface flag from the catalog data JSONB in the list queries (listCountries, listTracked). - Tracked rows in Browse now show a green check instead of a disabled empty checkbox; only untracked rows render the bulk-select checkbox. - 8px gap between country rows on Browse and Tracked. - Alias country_slug AS slug in the tracked-list query so row navigation, untrack, and keys resolve correctly.
…ete regulation fields) The detail page read invented field names (effective_date/description/url, timeline title) that don't exist in the feed, so effective dates, regulation detail, timeline text, and the narrative summaries rendered blank. ## Changes - Fix Regulation/TimelineEvent interfaces to the real feed fields. - Regulation cards now show type, effectiveDate, scope, key obligations, max penalty, industry tags, and the source link. - Timeline renders the real date + description. - New Overview section surfaces oneLiner, executiveSummary, and practicalTakeaway (stored but never displayed before).
…ings ## Changes Add DE/FR/ES translations for 18 UI-chrome strings introduced by the Regulations Tracker Horizon, Deadlines, and Frameworks pages: - Page titles: Horizon, Deadlines, International frameworks - Section labels: Key obligations, Key principles, Practical takeaway - Status labels: Scheduled, Not yet scheduled, Max penalty: - Subtitle/description strings for all three tabs - Empty-state messages (no changes/deadlines/frameworks recorded) - Stale-data notices (showing last known data; live unavailable) ## Benefits `npm run i18n:audit:strict` now reports 0 gaps for DE, FR, and ES (was 18 gaps each, 54 total).
Surfaces the three public feeds the app previously ignored, each as a new tab in the module sidebar with backend proxy + DB-cached fallback. ## Backend - Migration adds horizon/deadlines/frameworks JSONB columns to the meta singleton (global, non-tenant reference data). - Feed fetchers (fetchHorizon/fetchDeadlines/fetchSnapshot) + getGlobalFeed/ setGlobalFeeds utils. - GET /horizon, /deadlines, /frameworks endpoints: live feed first, fall back to the cached snapshot (stale flag) when the feed is unreachable. - Weekly sync refreshes all three (best-effort; never aborts the country sync). ## Frontend - Horizon (changelog), Deadlines (scheduled + unscheduled milestones), and Frameworks (11 international frameworks) pages. - Three new sidebar tabs + routes + active-tab detection. ## Note - Swagger/endpoints regenerated (680 ops, no drift).
## Changes - Backend: new `enrichWithFlags()` helper in `regulationsTracker.ctrl.ts` batches a single parameterized SQL lookup (`slug = ANY(:slugs)`) against the global `regulation_countries` catalog and attaches `countryFlag` to each deadline item; applied to both `deadlines` and `unscheduled` arrays on both the live-fetch and stored-fallback paths; best-effort (silently returns items unchanged if the catalog query fails) - Frontend: added `countryFlag?: string` to `Deadline` and `Unscheduled` interfaces; prepends the flag emoji inline with the country name in both the Scheduled and Not-yet-scheduled sections; falls back cleanly to name only when no flag is available ## Notes - Browse page already renders flags via CountryRowCard — no change needed; missing flags in local dev are a stale-seed issue fixed by a sync run - No new API routes added; no swagger regeneration required - 67/67 unit tests pass; Servers build, format-check, Clients typecheck and format-check all clean
…d where to manage keys
## Changes
- Backend: `getSettingsCtrl` now fetches the LLM key list once and derives two
additional non-secret fields — `llm_key_provider` (the `name` column of
`keys[0]`, e.g. "Anthropic") and `llm_key_model` (e.g. "gpt-4o") — alongside
the existing `has_llm_key` boolean. Both are `null` when no key exists. The
same key that `runImpactAnalysis` will use (most-recently-created, via
`ORDER BY created_at DESC`) is what is surfaced here.
- Frontend: the LLM-key status block on the Regulations Tracker Settings page
now branches on the key situation:
- No key: "Impact analysis needs an LLM key. Add an LLM key to turn this on."
with a SPA navigate link to /settings/apikeys.
- Key present: "Impact analysis will use your {provider} key ({model})."
with a "Manage keys" link to /settings/apikeys. Falls back to
"Impact analysis is active. Manage keys." if provider is unexpectedly null.
- i18n: added de/fr/es translations for the two new link strings ("Add an LLM
key", "Manage keys") so i18n:audit:strict passes at 100% coverage.
…lines Adds a horizontal 12-month runway strip above the Scheduled/Not-yet-scheduled lists on the Regulations Tracker Deadlines page. The strip shows each upcoming month as a grid column; deadline markers (flag emoji or brand-colored dot) are placed in the matching month column. Months within 90 days get a faint brand tint; the current month has a 2px left accent border. Clicking a marker scroll-jumps to the matching row in the Scheduled list with a 1.5s brand-outline highlight. Respects prefers-reduced-motion. New i18n strings added for de/fr/es.
Ran `generate:swagger` and `generate:endpoints` to pick up the two
impact-analysis routes added in the route layer but missing from the
generated artefacts:
GET /regulations-tracker/countries/{slug}/impact
POST /regulations-tracker/countries/{slug}/impact/refresh
`check:api-drift` now passes (683 Express endpoints = 683 Swagger ops).
…ages ## Changes - Add shared helper `statusVariant.ts` exporting `regulationStatusVariant()` mapping feed status values to semantic Chip variants: in-force → success, passed-not-active → info, proposed/draft → warning, policy-only/voluntary/unknown → default - CountryDetail: replace 15-line inline ternary chain with single helper call - Deadlines (scheduled + unscheduled): replace variant="default" with helper - Frameworks: replace variant="default" with helper for f.status chip; namedDocuments chips stay default (free-text, non-semantic) - Horizon: change-type chip (added/updated/status-change) set to variant="info" (visibly colored, not gray-on-gray; status helper not applicable here) ## Benefits - Single source of truth for status→colour mapping - All 5 feed status values now semantically coloured - Passes typecheck + format-check gates
…n heat tint, real tooltips on markers ## Changes - FIX 1: Remove 2px brand-green left border (today-marker rail) from the current month column entirely — no replacement marker. - FIX 2: Reduce urgency heat tint alpha from 0.06 to 0.035 for the first 3 months (≤90 days) — same logic, barely-there tint. - FIX 3: Replace native title= browser tooltips on deadline markers with VWTooltip (header = regulation name, content = date + country flag/name, placement=top). aria-label retained for a11y. Removed unused isCurrentMonth variable. Removed redundant title= attribute.
…scoverability callout ## Changes - Replace single-column stack of framework cards with a responsive CSS grid (1 col on xs, 2 col on md+) using MUI sx breakpoint object syntax - Tighten key-principles list item font/line-height (12px / 1.4) so half-width cards stay compact - Add subtle info callout above the grid: tells users the EU AI Act lives under each country, not in Frameworks; links to /regulations-tracker/browse via useNavigate (SPA navigation, no full reload) - Add "Find them in Browse." to de/fr/es dictionaries in translations.ts ## Benefits - Framework tab is easier to scan at a glance — two cards per row reduces excessive vertical scrolling - Users who expect the EU AI Act here are guided to Browse rather than hitting a dead end
Add an animated checklist panel inside the cadence-note box on the
Regulations Tracker Settings page that educates the user about what
the manual sync actually does while it runs.
## Changes
- `Settings/index.tsx`: local `SyncState` machine (idle/running/done/error)
drives four labelled stages: "Retrieving the latest regulations feed",
"Validating the feed", "Comparing against your tracked countries",
"Finishing up". Stages advance on a 800/850/900 ms simulation timer;
active stage shows a CircularProgress spinner (size 12), completed stages
show a green Check icon, upcoming stages are 40% opacity with a dot.
- Sim and real mutation run concurrently. Fast-forward: if the mutation
resolves before the sim reaches stage 3, the stage-3 timer reads
`mutationSettledRef` and resolves immediately. Hold-last-stage: if the
sim finishes first, stage 3 stays spinning until `onSuccess`/`onError`
fires. Error: clears all timers and transitions to error state inline.
- Done state shows the result counts ("Done — N changed, N removed") or
"Already up to date (N)." and a Dismiss button. Error state shows an
inline error message instead of the showError snackbar to avoid double-
notification. Button disabled while running.
- `i18n/translations.ts`: added 2 new strings (aria-label + error message)
in de/fr/es to keep i18n:audit:strict green.
## Benefits
- Informs users about what the sync actually does server-side (honest —
these stages genuinely run; only the per-stage timing is simulated).
- Keyboard-accessible (role=status, aria-live=polite); reduced-motion safe
(text labels + icons carry meaning, spinner is decoration only).
- No backend change; purely a frontend state machine.
…-enrichment ANY() syntax bug
The deadlines flag enrichment used 'slug = ANY(:slugs)', which Sequelize
expands to a comma list ANY('a','b') — a syntax error that the helper's
try/catch silently swallowed, so flags never appeared. Switched to IN (:slugs).
Added an idempotent migration backfilling data->>'flag' into existing
regulation_countries rows from the seed snapshot (older installs seeded before
flags were in the snapshot had null flags).
… A queries
## Changes
- Replace all 6 `= ANY(:param)` usages in getCandidates with `IN (:param)`,
which Sequelize expands correctly for JS array replacements.
`= ANY(:param)` produces `= ANY('a','b')` — a PostgreSQL syntax error
whenever the array has 2+ values.
- Fix pre-existing table name bug: `project_frameworks` → `projects_frameworks`
(matches the actual verifywise schema; caused the same runtime failure).
## Empty-array guards
- `:frameworks` (systems + controls): sentinel `["__none__"]` applied at
binding site when frameworks array is empty.
- `:frameworkExposure`: already guarded by mapFrameworksToExposure().
- `:projectIds` (assessments + vendors): gated by `if (candidateProjectIds.length)`
/ `candidateProjectIds.length > 0 ? ... : [-1]` — already safe.
- `:controlIds` (policies): gated by `if (controlIds.length)` — already safe.
## Verified
- IN with 1 value, 3 values, and __none__ sentinel all execute without error
against the real PostgreSQL DB.
- Empty array `IN ()` confirmed to error on this Sequelize version — sentinel
guard is mandatory.
## Tests
- 37/37 unit tests pass (36 existing + 1 new regression guard that asserts
no generated SQL contains `= ANY(`).
- `npm run build` clean.
…t, last-changed, tracked-since on tracked page
## Changes
- CountryRowCard: wrap the `checkbox` slot in a fixed 20×20 px flex container
so tracked rows (CheckSquare icon) and untracked rows (native input) always
start the card body at the same x-position; Tracked page (no checkbox) gets
an empty 20 px slot keeping the same alignment.
- CountryRow interface: add optional `regulation_count`, `last_changed_at`,
`created_at` fields (already returned by listTracked, dropped at the TS layer).
- CountryRowCard: add `showMeta` prop (default false). When true, renders a
secondary 12 px tertiary line showing "{n} regulation(s) · Last changed {date}
· Tracked since {date}", only for fields that are present.
- Tracked page: pass `showMeta` to CountryRowCard; fields flow through from the
existing useTracked() → data.data array without any API change.
- Browse page: unchanged — no showMeta prop passed, no metadata line rendered.
## Gates
- typecheck: clean
- i18n:audit:strict: clean (metadata built as computed JS strings, not static
JSX text nodes or prop literals — not scanned by the audit)
- format-check: clean
…new-count, controller SQL, guards, cleanup) ## Changes - FIX 1: Runway click-to-jump — build a `markerIdxMap` (deadline → array index) once in RunwayCalendar instead of calling `findIndex` per marker. findIndex always returned the first match on duplicate entries, jumping to the wrong row. Markers now carry the identical index used by Scheduled list row ids. - FIX 2 (backend): Add `newlyAdded: number` to syncRegulationsTracker return type (public + internal) and all return sites. The count was already computed from upsertFeedTx; now it surfaces to the caller so the Settings "check for updates" message `r.newlyAdded` is populated rather than undefined. Updated test assertion shape accordingly. - FIX 3: Move `enrichWithFlags` (raw SQL) from the controller into `regulationsTracker.utils.ts` — thin-controller convention. Removed the local copy and added an import. Also removed the now-unused `QueryTypes`/`sequelize` imports from the controller. - FIX 4: Parallelize the two enrichWithFlags calls in getDeadlines (live path and stored-fallback path) using Promise.all — they are independent DB queries. - FIX 5: Wrap `getSettings` call in `getImpactAnalysis` in its own try/catch; on failure, default to enabled (fail-open) so a transient settings blip does not convert this previously-settings-free GET into a 500. Updated test. - FIX 6: Guard `formatDate` in CountryRowCard.tsx — check `isNaN(d.getTime())` before calling toLocaleDateString; return null on bad input. Call sites guard the null with an `if (d)` before pushing to metaParts. - FIX 7: Replace hand-rolled clickable Box span (hardcoded #0F5A47) in Frameworks/index.tsx with `VWLink onClick=...` — proper theming, focus/keyboard semantics, and no hardcoded hex. - FIX 8 (no-op): Translation entries for RegulationsTracker strings kept. `npm run i18n:audit:strict` requires them (DOM-translator still needs the dictionary entries); removing them would cause coverage gaps. - FIX 9: Add explicit invariant comment to STAGE_DELAYS in Settings/index.tsx documenting that STAGE_DELAYS.length === SYNC_STAGES.length - 1. - NON-FIX: Add explanatory comment to `mapFrameworksToExposure` in regulationImpact.utils.ts noting why ISO 42001 / NIST AI RMF are absent from the mapping (no matching enum value in vendor.regulatory_exposure). ## Gates - `cd Servers && npm run build` ✓ - `npx jest syncRegulationsTracker|regulationImpact|regulationsTracker.ctrl` — 88/88 ✓ - `cd Clients && npm run typecheck` ✓ - `cd Clients && npm run i18n:audit:strict` — 0 gaps ✓ - `cd Clients && npm run format-check` ✓ - `cd Servers && npm run format-check` ✓
Redesigns the Timeline SectionCard from a disconnected bullet-dot list into a proper connected vertical timeline with: - A 1px absolute-positioned vertical rail (palette.border.dark) spanning the dot column, inset top/bottom by 6px so it starts/ends at dot centres. - Two-layer dot rings: outer border ring + inner filled dot. Latest event gets a brand-primary ring + brand dot (20px); older events get a muted border ring + border-dark dot (16px) for clear "most-recent" emphasis. - Dots use palette.background.main fill so the rail visually starts/ends at them rather than bleeding through. - Spacing expanded from 8px to 20px pb per row (breathing room for the rail); date typography shifted to brand-primary on latest event, darker secondary on older events; description lineHeight 1.6. - All spacing uses pixel strings; no hardcoded hex — palette tokens only. - Zero new dependencies; aria-hidden on decorative elements. Gates: typecheck clean, i18n:audit:strict 100% coverage, format-check clean.
…d and untracked The untracked row used a native <input type=checkbox> which renders larger than the tracked state's 16px CheckSquare icon, so ticked boxes looked smaller than unticked ones. Replace the native input with a button(role=checkbox) rendering the same 16px Lucide Square / CheckSquare icon, so both states share an identical box model and size, with keyboard focus support.
## Changes
- Cron `regulations_tracker_sync` moved from Mondays 06:00 UTC (`0 6 * * 1`)
to every day 06:00 UTC (`0 6 * * *`).
- Idempotency guard switched from ISO-week to UTC-day: new `currentIsoDay()`
helper (YYYY-MM-DD); `runSync` now skips only when the stored key equals
today. Without this, a daily cron would no-op Tue-Sun because the ISO week
was unchanged.
- Reuse the existing `last_run_week VARCHAR(10)` column to hold the day key
(fits "YYYY-MM-DD" exactly) — no migration. Renamed `clearLastRunWeek` ->
`clearLastRunDay`.
- Surface feed staleness: `getSettingsCtrl` now returns `feed_last_data_update`,
read from the stored horizon blob's `meta.lastDataUpdate` (the feed's own
data-update date, independent of when our sync last ran). Null-safe, no extra
external call. Lets the UI warn when the upstream feed itself goes stale —
previously indistinguishable from "nothing changed".
- Digest email subject de-weeklied ("Global AI regulations — update").
- Doc/comment drift fixed (weekly -> daily) across the module, the domain
reference doc, and the CLAUDE.md feature index; V1 limitation note updated to
flag the higher LLM fan-out under daily cadence.
## Benefits
- A regulation change is picked up — and tracked-country customers alerted —
the day it lands rather than waiting up to a week.
- No-change days stay cheap: hash-diff means 1 manifest fetch, 0 detail fetches.
## Notes
- Pre-existing: the day watermark commits inside upsertFeedTx before
notification dispatch, so a same-day dispatch failure delays the alert to the
next run. The daily cadence shrinks that window from ~7 days to ~1.
- Always pulls from verifywise.ai (servers are behind a firewall; no webhook,
no feed-origin override).
## Tests
- Added currentIsoDay unit tests + getSettingsCtrl feed_last_data_update tests;
updated the sync skip test to day semantics. tsc clean; 66/66 pass.
Adds six user-guide articles (browse, tracked, horizon, deadlines, frameworks, settings) covering every Regulations Tracker sidebar page plus impact analysis, and registers them in the content map and collection config. These pages referenced helpArticlePath ids that previously resolved to no content, so the in-app help drawer opened empty.
…wse controls by role Tracking (track / untrack / bulk-track) was restricted to Admin and SuperAdmin on the backend, and the Browse page showed the controls to every user, so ineligible roles got a failing click. Tracking is now allowed for Admin, SuperAdmin, and Editor via a canTrack helper on the three track endpoints (Settings, sync, and refresh stay admin-only). The Browse page now gates the per-row Track/Untrack button, the row checkbox, and the bulk-track toolbar behind the same canTrack role check, so other roles see a read-only catalogue. Updated the controller role tests and the Browse user-guide article.
The MJML digest header still read 'Global AI regulations — weekly update'
after the weekly->daily cadence switch (the .mjml template was missed by the
earlier .ts-only sweep). Aligns it with the code-side subject ('— update')
so daily change alerts are not mislabelled as weekly.
…y drill-down ## Changes - Empty-state copy on Horizon, Frameworks and Deadlines reframed from terse 'no data recorded yet' (reads as broken) to reassuring 'you're all caught up / will appear here once published' guidance. - Deadlines rows (Scheduled + Not-yet-scheduled) now link the country name to its detail page (/regulations-tracker/<slug>) with hover affordance + keyboard access, closing the dead-end where deadlines couldn't reach country detail. ## Benefits - New/quiet orgs no longer see pages that look broken on first open. - Deadlines becomes a navigable entry point into full country detail, not a read-only list.
The per-org impact summary (which of YOUR systems/controls/policies/vendors a
change affects) was only ever appended to the in-app notification message — the
smallest surface — while the spacious email digest never carried it. This puts
the verbose, valuable content where there's room for it.
## Changes
- New impactSectionMjml() renders a 'How these changes affect your organization'
block in the email: per affected country, each entity group (systems, controls,
policies, vendors, assessments) listed with the specific item name AND the
LLM's one-line reason. Empty groups are omitted; the whole section is omitted
when nothing is affected or impact is off/keyless.
- renderDigest() now takes an optional per-country impact list; new
{{impactSection}} placeholder added to the MJML template after the removed
section.
- Impact results computed during the in-app loop are captured in impactBySlug
and reused for the email. Email-only orgs (no in-app recipients) now also get
impact: a backfill pass runs analysis for changed countries not yet analyzed,
honoring the same LLM-key/enabled gate and the per-run IMPACT_MAX_ANALYSES cap.
- In-app notifications keep the short counts-only suffix (right for a cramped
surface); the email gets the detail.
## Tests
- tsc clean; existing syncRegulationsTracker suite (10 tests) passes.
…g an org A super-admin is org-less and gets read-only access when viewing any organization (the backend blocks all non-GET writes with 403). The Regulations Tracker UI was showing them write controls that would fail: the Browse track/untrack/bulk buttons, the country-detail Track button, the impact Re-analyse button, and the editable Settings view. Each now excludes super-admins (and keeps tracking to admins/editors, re-analyse and settings to admins), so a super-admin sees the module read-only. Updated the Browse user-guide article to note this.
…try to the sync ## Changes - Log a per-run feed-diff quality split: how many changed countries arrived with a structured field-level diff vs. hash-only (no structured diff). The hash-only case is also warned with the affected slugs, since those produce generic impact verdicts. - Tally impact-analysis outcomes (ok / skipped_no_candidates / error / no_key / cached) across every org x country pass via a new tallyImpact() helper, and emit them in an end-of-run 'value telemetry' log line. ## Benefits - Makes the real value of the LLM-enriched digest measurable in production: whether the upstream feed actually publishes change-specific diffs, and how often the impact pass yields a usable verdict versus having nothing to judge.
… structured diff ## Problem When the feed moved a country's hash but carried no structured field-level diff (unstructured change), the impact pass still ran. With nothing concrete to judge, the LLM was told '(no structured diff available)' and produced a generic verdict — a misleading 'how this affects you' block built on no real change. This is the current live-feed state for every country, so admins would get generic impact panels rather than change-specific value. ## Changes - Thread the per-country 'unstructured' flag from the feed diff into the notification/email buckets. - Gate both impact-analysis call sites (in-app pass and email backfill) on !unstructured: no diff -> no LLM call, no impact panel, no LLM spend. - Suppress the 'configure an LLM key' nudge for unstructured changes too, since a key would not have produced a panel there either. - The plain change notification and the 'Changed' email section still fire so the org always knows the country changed. ## Benefits - The enriched impact block now appears only when there is a real, specific change to reason about — so when it shows, it carries genuine signal. - Avoids wasted LLM calls on no-op hash moves. ## Tests - Added a case asserting an unstructured change makes no runImpactAnalysis call, still notifies (no 'Impact:' suffix, no key nudge), and emails with an empty impact section. Full suite: 11 passing.
…crumbs and Activity tab
## Changes
- Frameworks: equal-width, responsive two-column grid (repeat(2, minmax(0, 1fr))
+ align-items: start) so a long unbreakable string can no longer widen one
column. Long named-document tags now wrap inside the card instead of
overflowing the page (replaced the non-wrapping Chip with a wrapping pill).
- Renamed the "Horizon" tab to "Activity" (sidebar label + page title). The page
is a backward-looking changelog, so "Horizon" (which reads as future-looking)
was misleading.
- Tracked: removed the empty leading checkbox slot so flags sit flush against the
card padding (the slot now only renders when a checkbox is supplied; Browse,
which passes one, is unchanged).
- Breadcrumbs: added route label + icon mappings for every regulations-tracker
tab and a country-detail pattern, so the second crumb is split (e.g. "Browse"
instead of "Regulations Tracker / Bro...") and every crumb shows an icon.
- Reworded the Deadlines empty-state ("they are published") and added the new
user-facing strings to de/fr/es. i18n strict: 0 gaps.
## Benefits
- Consistent, responsive layout with no horizontal overflow.
- Clearer navigation and accurate tab wording.
…ons to the changed regulation
## Problem
buildContext built the LLM impact prompt with two flaws:
1. It matched the wrong change field names ("status" / "effectiveDate") instead
of the feed shape ("regulation.status" / "regulation.effectiveDate"), so
status and effective-date changes were silently dropped — the LLM never saw
them.
2. It flattened obligations from every regulation in the country (46 lines for
the US), drowning the few duties that actually changed and working against
the "judge the change, not the regulation" instruction.
## Changes
- Match the feed RegulationChange shape exactly, and include the changed
regulation name in each status/effective-date change line.
- Collect the names of the regulation(s) that changed and scope obligations to
just those (matched against regs[].name). Fall back to all obligations when no
specific regulation can be identified (e.g. a regulationCount-only change or an
added regulation whose value does not line up), so the prompt is never empty.
## Benefits
- The impact analysis now sees the real, full change set, and the prompt
foregrounds the new/altered duties instead of the whole regulatory baseline —
sharper verdicts and fewer tokens.
## Tests
- Added buildContext coverage: correct field capture, obligation scoping,
count-only fallback, added-regulation matching, and missing-history. 45 tests
passing across the impact + sync suites.
…d migrations from review Addresses verified findings from a full-branch expert review. ## Sync pipeline - Replace the module-level syncInProgress boolean with a Postgres advisory lock (acquireSyncLock) so concurrency is enforced across worker processes/pods, not just within one Node process. Keep the boolean as a cheap same-process fast-path. Fail open if the lock machinery errors. - Isolate each org's notification/email dispatch in its own try/catch so one org failure no longer aborts the loop and starves every subsequent org; record a "partial" run status when any org fails. - Remove queue.obliterate from the vendor and report schedulers. It wiped the ENTIRE shared BullMQ queue, making job survival order-dependent and at risk of silently dropping the regulations-tracker daily job. Repeatable adds are idempotent by repeat key. Drop the now-stale ordering comment. ## Impact analysis - validateVerdicts: coerce numeric-string ids that some models emit instead of silently dropping the verdict; the sentKeys guard still rejects hallucinated ids. Added tests. - parseJsonLoose: scan brace depth to find the first top-level object instead of lastIndexOf, which mis-bounds when the model appends prose containing a brace. ## Controller - GET /frameworks no longer writes setGlobalFeeds: a GET must not mutate shared cross-tenant state; the daily sync owns that cache write. Drop the now-unused import. - Return a generic "Internal server error" from all 14 500 responses instead of the raw error message (which could leak SQL/schema detail). Detail is still logged via logFailure. ## Migrations - Widen regulation_tracker_meta.last_run_week VARCHAR(10) to VARCHAR(20). - Drop the redundant idx_reg_impact_org_slug index (the UNIQUE org+slug constraint already creates an identical index). - Seed down: stop the bare DELETE FROM regulation_countries that would wipe the global catalog for all tenants on a seed-only rollback; reset markers only. - Guard fs.readFileSync in the seed (throw with a clear message) and the flag backfill (skip gracefully) when the snapshot file is absent. - Use ALTER TABLE IF EXISTS in the impact-table down for out-of-order rollback. ## Frontend - Settings: drop the dead SuperAdmin branch in isAdmin (unreachable behind !isSuperAdmin). ## Notes - getAffectedOrgsBySlugs intentionally keeps no is_active filter (removed countries must still notify); documented to prevent a wrong "fix". All gates green: server build, 47 server tests, client typecheck, format-check.
## Problem The Tracked/Browse/Deadlines pages showed no country flags. The manifest feed does NOT carry a per-country flag (only the detail endpoint does, inconsistently), so every daily sync overwrote regulation_countries.data with flag-less manifest entries — wiping the one-time backfill migration and leaving data->>flag NULL for all 60 countries. ## Fix Treat the committed seed snapshot as the durable source of truth for flags. Add a lazily-loaded slug -> emoji map and re-inject the flag into every row stored by upsertFeedTx (withFlag), so flags survive every sync regardless of feed shape. Presentation-only — never part of the change-detection hash. ## Benefits - Flags now render and stay rendered after each daily sync. - No dependency on the feed providing flags. Verified: build clean, 11 sync tests pass, flags render on the Tracked page. Existing local rows were repaired in place; new installs get flags via the seed + this injection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Regulations Tracker module — a window into the public Global AI Regulations feed — plus an impact analysis layer that maps regulation changes to an organisation's own governance entities.
Regulations tracker (base module)
Impact analysis (detect-only)
Notes