From fb2f98cae58a01f021c3f611bb16e3d5a067164a Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 09:37:14 +0200 Subject: [PATCH 01/15] test: promote the phase layers into the gate, and open the JS side Three strands, all the same idea: make the size of what is NOT measured visible, then shrink it. 1. THE PHASE LAYERS ARE IN THE GATE. EntraID/Omada/midPoint Phases moved from `exclusions` to `mutate` (109 -> 112 files, 23 -> 20 exclusions). Their exclusion reasons had gone stale in the way this whole effort is about: they still asserted 39.6% / 57.8% / 55.9% when the files now measure 64.3% / 65.8% / 63.0%. A written reason that has quietly stopped being true is worse than no reason, because it reads as considered. Deleting them removes that. 2. THE JS BACKLOG EXISTS NOW. .ci/js-mutation-scope-baseline.json lists the 402 eligible .js/.jsx files with no mutation evidence, and app/api/src/mutationScope.guard.test.js enforces that every eligible file is mutation-tested, excluded with a written reason, or on that list -- which may only SHRINK. This is the PowerShell scope guard's lesson applied before the same thing happens here: 4 of 409 files were covered, and nothing said so. Two properties learned the hard way over there: the guard asserts it found >300 files BEFORE concluding anything, because an empty walk satisfies "every file is decided" vacuously -- which is precisely how the PowerShell version reported full coverage of a decision nobody had made. And it fails on stale or already-covered entries, so the backlog cannot drift into overstating the gap. 3. EFFECTIVE ACCESS BROUGHT IN, 79.3% -> ~93% across the scope. policies.js 83.6% -> 96.4%: every survivor sat in the logic picking the DECISIVE ace, which drives the Direct/Indirect badge. They survived because every fixture listed the expected winner LAST, so "keep whichever ace we just looked at" gave the same answer as "pick the best one" -- the reduce could have ignored its accumulator entirely and passed. Putting the winner first is the only ordering that separates them. That also hid a real distinction: an explicit group grant and an inherited one both badge Indirect, so badge-only assertions could not see rank at all. engine.js 69.4% -> 91.6%: - `explicit: r.holder === principalId` had no fixture where a GROUP held the grant, so hard-coding it true survived. That flag is the Direct badge: a user would be told they hold access directly while the group it actually came from vanishes from the answer. - both truncation caps were untested. depth >= maxDepth and depthByNode.size >= maxNodes decide when an access answer comes back INCOMPLETE, and each is one off-by-one from silently returning less access than the principal has. The fixture chain is 2 deep, so maxDepth 1 and a node budget of 2 land exactly where >= and > disagree. - a diamond graph re-admitted its shared node; on a cyclic graph the walk would not terminate. - the per-resolve telemetry line hard-coded cacheHit, so nothing showed whether the cache did anything at all. lru.js is new, extracted from engine.js at 94.7% (100% of killable). Inside the engine it was reachable only through resolve() with a database and enough distinct keys to force an eviction, so NOTHING exercised recency, eviction or capacity. Same move as the PSMutant orchestrator: code that cannot be measured where it sits gets lifted somewhere it can. Remaining survivors checked and equivalent: map.delete() on an absent key is a no-op, and `r < rb -> r <= rb` sits inside `if (r !== rb)`. TWO GATES CAUGHT ME, both correctly: - the inline db-mock ratchet, because my new test added a 61st inline factory. Switched to the shared manual mock -- and then the ratchet still failed, because my COMMENT explaining the rule quoted the offending call and the check matches on file text. Reworded to describe the pattern instead. - three unrelated tests timed out at 5s during the full run. They take 11-22s under the CPU load of a concurrent mutation run and pass in isolation (97/97); not caused by this change. Suites: effectiveAccess 45 -> 76 tests, scope guard 5, all passing. --- .ci/js-mutation-scope-baseline.json | 407 ++++++++++++++ .ci/psmutant.config.json | 509 +++++++++++++----- app/api/src/effectiveAccess/engine.js | 28 +- .../effectiveAccess/engineContainment.test.js | 38 ++ .../engineObservability.test.js | 159 ++++++ app/api/src/effectiveAccess/lru.js | 35 ++ app/api/src/effectiveAccess/lru.test.js | 114 ++++ app/api/src/effectiveAccess/policies.test.js | 59 ++ app/api/src/mutationScope.guard.test.js | 121 +++++ app/api/stryker.effectiveaccess.config.json | 36 ++ .../vitest.stryker.effectiveaccess.config.js | 23 + changes/phases-into-gate.md | 5 + 12 files changed, 1364 insertions(+), 170 deletions(-) create mode 100644 .ci/js-mutation-scope-baseline.json create mode 100644 app/api/src/effectiveAccess/engineObservability.test.js create mode 100644 app/api/src/effectiveAccess/lru.js create mode 100644 app/api/src/effectiveAccess/lru.test.js create mode 100644 app/api/src/mutationScope.guard.test.js create mode 100644 app/api/stryker.effectiveaccess.config.json create mode 100644 app/api/vitest.stryker.effectiveaccess.config.js create mode 100644 changes/phases-into-gate.md diff --git a/.ci/js-mutation-scope-baseline.json b/.ci/js-mutation-scope-baseline.json new file mode 100644 index 000000000..21bb95936 --- /dev/null +++ b/.ci/js-mutation-scope-baseline.json @@ -0,0 +1,407 @@ +{ + "_comment": "Grandfathered JavaScript files that are eligible for mutation testing but have not been triaged yet. The PowerShell side has the same list (.ci/psmutant-scope-baseline.json) and it is what took that backlog from an invisible 103 files to zero: without an explicit list, 'the mutation gate is green' and 'almost nothing is in the mutation gate' look identical. Today 7 of 409 eligible JS files have mutation evidence -- app/api/src/auth (readTokens, permissions) via stryker.auth.config.json, and app/ui/src (usePermissions, matrixFilter) via stryker.pilot.config.json, and app/api/src/effectiveAccess (engine, engine.helpers, policies) via stryker.effectiveaccess.config.json -- so this list is the other 402. ELIGIBLE means: a .js/.jsx file under app/api/src or app/ui/src that is not a test (*.test.*, *.spec.*) and not under __tests__/, test-utils/, mock/, node_modules/, coverage/, dist/ or build/. THE LIST MAY ONLY SHRINK. A file leaves it by being added to a Stryker config's `mutate` list, or by being given a written exclusion reason in that config -- the same two exits the PowerShell files had. A NEW file may never be added: the guard test fails on any eligible file that is neither mutation-tested, excluded, nor already listed here, so new code has to be decided on when it lands rather than accumulating quietly. This is a backlog, not a plan: nothing here is committed to a date, and some of it will end up excluded rather than tested (pure JSX page shells, for instance, where mutating markup says little about behaviour). What it buys is that the size of the gap is a number somebody can see.", + "grandfathered": [ + "app/api/src/accountlinking/classifier.js", + "app/api/src/accountlinking/defaultRules.js", + "app/api/src/accountlinking/engine.helpers.js", + "app/api/src/accountlinking/engine.js", + "app/api/src/app.js", + "app/api/src/auth/permissionManifest.js", + "app/api/src/bootstrap.js", + "app/api/src/cli/auth-config.js", + "app/api/src/config/authConfig.js", + "app/api/src/contexts/contextFilters.js", + "app/api/src/contexts/cycleGuard.js", + "app/api/src/contexts/memberCounts.js", + "app/api/src/contexts/plugins/ad-ou-from-dn.js", + "app/api/src/contexts/plugins/department-from-principal.js", + "app/api/src/contexts/plugins/entra-group-category-tree.js", + "app/api/src/contexts/plugins/manager-hierarchy.helpers.js", + "app/api/src/contexts/plugins/manager-hierarchy.js", + "app/api/src/contexts/plugins/orphaned-accounts.js", + "app/api/src/contexts/plugins/principal-type-tree.js", + "app/api/src/contexts/plugins/registry.js", + "app/api/src/contexts/plugins/resource-cluster/index.js", + "app/api/src/contexts/plugins/resource-cluster/tokenize.js", + "app/api/src/contexts/plugins/resource-type-tree.js", + "app/api/src/contexts/plugins/risky-consent.js", + "app/api/src/contexts/plugins/riskyAppFeed.js", + "app/api/src/contexts/plugins/riskyConsentRiskMap.js", + "app/api/src/contexts/plugins/runner.helpers.js", + "app/api/src/contexts/plugins/runner.js", + "app/api/src/contexts/plugins/scope-hierarchy.js", + "app/api/src/contexts/plugins/types.js", + "app/api/src/contexts/seedAlgorithms.js", + "app/api/src/crawlerManifests.js", + "app/api/src/db/__mocks__/connection.js", + "app/api/src/db/columnCache.js", + "app/api/src/db/connection.js", + "app/api/src/db/matrixHelpers.js", + "app/api/src/db/migrate.js", + "app/api/src/db/queryHelpers.js", + "app/api/src/db/schemaErrors.js", + "app/api/src/db/sqlParams.js", + "app/api/src/docs/docsStructure.js", + "app/api/src/export/excelWorkbook.js", + "app/api/src/export/exportBaseUrl.js", + "app/api/src/export/queryTemplates.js", + "app/api/src/index.js", + "app/api/src/ingest/crawlerPresence.js", + "app/api/src/ingest/engine.js", + "app/api/src/ingest/normalization.js", + "app/api/src/ingest/sessions.js", + "app/api/src/ingest/tempTableHelpers.js", + "app/api/src/ingest/tombstonePurge.js", + "app/api/src/ingest/validation.helpers.js", + "app/api/src/ingest/validation.js", + "app/api/src/lib/capabilityId.js", + "app/api/src/lib/jsonb.js", + "app/api/src/lib/listParams.js", + "app/api/src/lib/listSort.js", + "app/api/src/lib/principalTypes.js", + "app/api/src/lib/referenceFilters.js", + "app/api/src/lib/ssrfGuard.js", + "app/api/src/lib/syncVersion.js", + "app/api/src/llm/providers.js", + "app/api/src/llm/riskPrompts.js", + "app/api/src/llm/scraper.js", + "app/api/src/llm/service.js", + "app/api/src/matrix/attrExpr.js", + "app/api/src/matrix/attributeCut.js", + "app/api/src/matrix/contextRollup.js", + "app/api/src/matrix/filterSql.js", + "app/api/src/matrix/inheritedAccess.js", + "app/api/src/matrix/resourceContexts.js", + "app/api/src/matrix/rollupBuilders.js", + "app/api/src/matrix/scopeHistory.js", + "app/api/src/middleware/auth.js", + "app/api/src/middleware/crawlerAuth.helpers.js", + "app/api/src/middleware/crawlerAuth.js", + "app/api/src/middleware/perfMetrics.js", + "app/api/src/perf/collector.js", + "app/api/src/perf/sqlTimer.js", + "app/api/src/postCrawlJobs.js", + "app/api/src/riskscoring/engine.helpers.js", + "app/api/src/riskscoring/engine.js", + "app/api/src/riskscoring/tiers.js", + "app/api/src/routes/accountLinking.js", + "app/api/src/routes/admin.js", + "app/api/src/routes/admin/curatedData.js", + "app/api/src/routes/admin/curatedExport.js", + "app/api/src/routes/admin/curatedImport.js", + "app/api/src/routes/admin/dashboard.js", + "app/api/src/routes/admin/maintenance.js", + "app/api/src/routes/admin/riskConfig.js", + "app/api/src/routes/admin/settings.js", + "app/api/src/routes/authRoles.js", + "app/api/src/routes/bulkLists.js", + "app/api/src/routes/categories.js", + "app/api/src/routes/contextPlugins.js", + "app/api/src/routes/contexts.js", + "app/api/src/routes/contexts/crud.js", + "app/api/src/routes/contexts/crudHelpers.js", + "app/api/src/routes/contexts/members.js", + "app/api/src/routes/contexts/read.js", + "app/api/src/routes/contexts/shared.js", + "app/api/src/routes/crawlerFiles.js", + "app/api/src/routes/crawlers.js", + "app/api/src/routes/crawlers/admin.js", + "app/api/src/routes/crawlers/selfService.js", + "app/api/src/routes/crawlers/shared.js", + "app/api/src/routes/dataExport.js", + "app/api/src/routes/details.js", + "app/api/src/routes/details/accessPackage.js", + "app/api/src/routes/details/accessPackageDetail.js", + "app/api/src/routes/details/group.js", + "app/api/src/routes/details/groupDetail.js", + "app/api/src/routes/details/shared.js", + "app/api/src/routes/details/user.js", + "app/api/src/routes/details/userDetail.js", + "app/api/src/routes/effectiveAccess.js", + "app/api/src/routes/governance.js", + "app/api/src/routes/identities.js", + "app/api/src/routes/identities/detail.js", + "app/api/src/routes/identities/detailData.js", + "app/api/src/routes/identities/list.js", + "app/api/src/routes/identities/listQuery.js", + "app/api/src/routes/identities/overrides.js", + "app/api/src/routes/identities/shared.js", + "app/api/src/routes/ingest.js", + "app/api/src/routes/ingest/handlers.js", + "app/api/src/routes/ingest/helpers.js", + "app/api/src/routes/ingest/matrixViews.js", + "app/api/src/routes/jobs.js", + "app/api/src/routes/jobs/configs.js", + "app/api/src/routes/jobs/helpers.js", + "app/api/src/routes/jobs/runs.js", + "app/api/src/routes/llm.js", + "app/api/src/routes/matrix.js", + "app/api/src/routes/matrix/data.js", + "app/api/src/routes/matrix/savedFilters.js", + "app/api/src/routes/matrix/scope.js", + "app/api/src/routes/matrix/shared.js", + "app/api/src/routes/orgChart.js", + "app/api/src/routes/perf.js", + "app/api/src/routes/permissions.js", + "app/api/src/routes/permissions/accessPackages.js", + "app/api/src/routes/permissions/grid.js", + "app/api/src/routes/permissions/gridQuery.js", + "app/api/src/routes/permissions/nestedGroups.js", + "app/api/src/routes/permissions/shared.js", + "app/api/src/routes/permissions/syncLog.js", + "app/api/src/routes/permissions/userColumns.js", + "app/api/src/routes/preferences.js", + "app/api/src/routes/recentChanges.js", + "app/api/src/routes/recentChanges/changes.js", + "app/api/src/routes/recentChanges/classify.js", + "app/api/src/routes/recentChanges/shared.js", + "app/api/src/routes/recentChanges/timeline.js", + "app/api/src/routes/resources.js", + "app/api/src/routes/resources/detail.js", + "app/api/src/routes/resources/list.js", + "app/api/src/routes/riskProfiles.js", + "app/api/src/routes/riskProfiles/helpers.js", + "app/api/src/routes/riskScores.js", + "app/api/src/routes/riskScores/entity.js", + "app/api/src/routes/riskScores/list.js", + "app/api/src/routes/riskScores/shared.js", + "app/api/src/routes/riskScores/summary.js", + "app/api/src/routes/riskScoringRuns.js", + "app/api/src/routes/systems.js", + "app/api/src/routes/tags.js", + "app/api/src/routes/tags/crud.js", + "app/api/src/routes/tags/entities.js", + "app/api/src/routes/tags/shared.js", + "app/api/src/routes/updates.js", + "app/api/src/scheduler.js", + "app/api/src/secrets/crawlerSecrets.js", + "app/api/src/secrets/migrateCrawlerSecrets.js", + "app/api/src/secrets/vault.js", + "app/api/src/startupState.js", + "app/api/src/updates/channel.js", + "app/api/src/updates/checkForUpdates.js", + "app/api/src/updates/componentVersions.js", + "app/api/src/updates/detect.js", + "app/api/src/updates/job.js", + "app/api/src/updates/versionCompare.js", + "app/api/src/version.js", + "app/ui/src/App.helpers.js", + "app/ui/src/App.jsx", + "app/ui/src/auth/AuthGate.js", + "app/ui/src/auth/AuthGateProvider.jsx", + "app/ui/src/components/AboutPage.jsx", + "app/ui/src/components/AccessPackageDetailPage.jsx", + "app/ui/src/components/AccessPackageGovernance.jsx", + "app/ui/src/components/AccessPackagesPage.jsx", + "app/ui/src/components/AccountLinkingSettings.jsx", + "app/ui/src/components/AdminPage.jsx", + "app/ui/src/components/AuthSettingsPage.jsx", + "app/ui/src/components/ConfidenceBar.jsx", + "app/ui/src/components/ContextAttributesTab.jsx", + "app/ui/src/components/ContextDetailHeader.jsx", + "app/ui/src/components/ContextDetailPage.jsx", + "app/ui/src/components/ContextRelationshipsTab.jsx", + "app/ui/src/components/ContextsPage.jsx", + "app/ui/src/components/CopyButton.jsx", + "app/ui/src/components/CrawlersPage.helpers.js", + "app/ui/src/components/CrawlersPage.jsx", + "app/ui/src/components/DashboardComposeWarning.jsx", + "app/ui/src/components/DashboardFeatureRow.jsx", + "app/ui/src/components/DashboardPage.jsx", + "app/ui/src/components/DashboardStatsPanel.jsx", + "app/ui/src/components/DashboardTrendsTab.jsx", + "app/ui/src/components/DeletedBadge.jsx", + "app/ui/src/components/DepartmentBadges.jsx", + "app/ui/src/components/DepartmentDetailPage.jsx", + "app/ui/src/components/DepartmentHeader.jsx", + "app/ui/src/components/DetailSection.jsx", + "app/ui/src/components/DialogProvider.jsx", + "app/ui/src/components/EmptyState.jsx", + "app/ui/src/components/EntityDetailLayout.jsx", + "app/ui/src/components/EntityDetailPage.jsx", + "app/ui/src/components/EntityGraph.jsx", + "app/ui/src/components/EntityListPage.jsx", + "app/ui/src/components/EntityTimeline.jsx", + "app/ui/src/components/ErrorBoundary.jsx", + "app/ui/src/components/ExpandedItemsList.helpers.js", + "app/ui/src/components/ExpandedItemsList.jsx", + "app/ui/src/components/FilterBar.jsx", + "app/ui/src/components/GovernancePage.jsx", + "app/ui/src/components/GroupsPage.jsx", + "app/ui/src/components/IdentitiesPage.jsx", + "app/ui/src/components/IdentityDetailPage.jsx", + "app/ui/src/components/JobPhasesModal.helpers.js", + "app/ui/src/components/JobPhasesModal.jsx", + "app/ui/src/components/JsonViewer.jsx", + "app/ui/src/components/LinkedAccountsPanel.jsx", + "app/ui/src/components/MappingRows.jsx", + "app/ui/src/components/MatrixView.jsx", + "app/ui/src/components/MembersSection.jsx", + "app/ui/src/components/PerfPage.jsx", + "app/ui/src/components/PluginsPage.jsx", + "app/ui/src/components/RecentChangesSection.helpers.jsx", + "app/ui/src/components/RecentChangesSection.jsx", + "app/ui/src/components/ResourceDetailPage.constants.js", + "app/ui/src/components/ResourceDetailPage.jsx", + "app/ui/src/components/RiskProfileClassifiersStep.jsx", + "app/ui/src/components/RiskProfileProgressPanel.jsx", + "app/ui/src/components/RiskProfileRefineStep.jsx", + "app/ui/src/components/RiskProfileSaveStep.jsx", + "app/ui/src/components/RiskProfileScoringStep.jsx", + "app/ui/src/components/RiskProfileSourcesStep.jsx", + "app/ui/src/components/RiskProfileWizard.jsx", + "app/ui/src/components/RiskScoreSection.constants.js", + "app/ui/src/components/RiskScoreSection.helpers.js", + "app/ui/src/components/RiskScoreSection.jsx", + "app/ui/src/components/RiskScoringPage.jsx", + "app/ui/src/components/RiskSummary.jsx", + "app/ui/src/components/RolesPermissionsSection.jsx", + "app/ui/src/components/RollupMatrixView.jsx", + "app/ui/src/components/RotatedMatrixView.helpers.js", + "app/ui/src/components/RotatedMatrixView.jsx", + "app/ui/src/components/RunDetailPage.jsx", + "app/ui/src/components/ScheduleEditor.jsx", + "app/ui/src/components/Spinner.jsx", + "app/ui/src/components/Stepper.jsx", + "app/ui/src/components/SyncLogPage.jsx", + "app/ui/src/components/SystemsPage.jsx", + "app/ui/src/components/TabBar.jsx", + "app/ui/src/components/TimeSeriesChart.jsx", + "app/ui/src/components/UpdatesSettings.jsx", + "app/ui/src/components/UserDetailPage.jsx", + "app/ui/src/components/UsersPage.jsx", + "app/ui/src/components/WizardShell.jsx", + "app/ui/src/components/accessPackages/AccessPackageRow.jsx", + "app/ui/src/components/accessPackages/AccessPackagesFilterBar.jsx", + "app/ui/src/components/accessPackages/AccessPackagesHeader.jsx", + "app/ui/src/components/accessPackages/AccessPackagesPagination.jsx", + "app/ui/src/components/accessPackages/AccessPackagesTable.jsx", + "app/ui/src/components/accessPackages/CategoryManagementBar.jsx", + "app/ui/src/components/accessPackages/ComplianceStatusCell.jsx", + "app/ui/src/components/accessPackages/CreateCategoryForm.jsx", + "app/ui/src/components/accessPackages/ReviewedByCell.jsx", + "app/ui/src/components/accessPackages/RowCategorySelect.jsx", + "app/ui/src/components/accessPackages/SelectionActionBar.jsx", + "app/ui/src/components/admin/CuratedDataSection.jsx", + "app/ui/src/components/admin/DangerZoneSection.jsx", + "app/ui/src/components/admin/HistoryRetentionSection.jsx", + "app/ui/src/components/admin/LLMActionButtons.jsx", + "app/ui/src/components/admin/LLMApiKeyField.jsx", + "app/ui/src/components/admin/LLMAzureFields.jsx", + "app/ui/src/components/admin/LLMModelField.jsx", + "app/ui/src/components/admin/LLMProviderField.jsx", + "app/ui/src/components/admin/LLMSettingsSection.helpers.js", + "app/ui/src/components/admin/LLMSettingsSection.jsx", + "app/ui/src/components/admin/LLMStatusMessages.jsx", + "app/ui/src/components/admin/LLMTestResult.jsx", + "app/ui/src/components/admin/PowerQueryExportSection.jsx", + "app/ui/src/components/admin/RiskScoringSection.jsx", + "app/ui/src/components/admin/adminFormat.js", + "app/ui/src/components/admin/adminIcons.jsx", + "app/ui/src/components/admin/adminTabs.js", + "app/ui/src/components/admin/adminUi.jsx", + "app/ui/src/components/app/AppFooter.jsx", + "app/ui/src/components/app/AppHeader.jsx", + "app/ui/src/components/app/AppMain.jsx", + "app/ui/src/components/app/AppNav.jsx", + "app/ui/src/components/app/BackendErrorScreen.jsx", + "app/ui/src/components/app/Brand.jsx", + "app/ui/src/components/app/DetailRoute.jsx", + "app/ui/src/components/app/DetailTab.jsx", + "app/ui/src/components/app/MatrixArea.jsx", + "app/ui/src/components/app/NavTabButton.jsx", + "app/ui/src/components/app/SettingsMenu.jsx", + "app/ui/src/components/app/ThemeSelector.jsx", + "app/ui/src/components/contexts/ContextListView.jsx", + "app/ui/src/components/contexts/ContextMemberPicker.jsx", + "app/ui/src/components/contexts/ContextPicker.helpers.js", + "app/ui/src/components/contexts/ContextPicker.jsx", + "app/ui/src/components/contexts/ContextTreeSelector.jsx", + "app/ui/src/components/contexts/ContextTreeView.helpers.js", + "app/ui/src/components/contexts/ContextTreeView.jsx", + "app/ui/src/components/contexts/ManualContextEditor.helpers.js", + "app/ui/src/components/contexts/ManualContextEditor.jsx", + "app/ui/src/components/contexts/ModalPrimitives.jsx", + "app/ui/src/components/contexts/NewContextSchemaForm.jsx", + "app/ui/src/components/contexts/NewContextWizard.helpers.js", + "app/ui/src/components/contexts/NewContextWizard.jsx", + "app/ui/src/components/contexts/NewContextWizardBody.jsx", + "app/ui/src/components/contexts/NewContextWizardFooter.jsx", + "app/ui/src/components/departmentTiers.js", + "app/ui/src/components/dialogContext.js", + "app/ui/src/components/entityGraphShape.js", + "app/ui/src/components/inputs/ChevronDown.jsx", + "app/ui/src/components/inputs/Combobox.jsx", + "app/ui/src/components/inputs/Select.jsx", + "app/ui/src/components/matrix/AttributePicker.jsx", + "app/ui/src/components/matrix/ContextFilterControl.jsx", + "app/ui/src/components/matrix/InheritancePathModal.jsx", + "app/ui/src/components/matrix/MatrixAggregateNameCell.jsx", + "app/ui/src/components/matrix/MatrixApBandCell.jsx", + "app/ui/src/components/matrix/MatrixApLabelCell.jsx", + "app/ui/src/components/matrix/MatrixCell.helpers.js", + "app/ui/src/components/matrix/MatrixCell.jsx", + "app/ui/src/components/matrix/MatrixColumnHeaders.helpers.js", + "app/ui/src/components/matrix/MatrixColumnHeaders.jsx", + "app/ui/src/components/matrix/MatrixContextsCell.jsx", + "app/ui/src/components/matrix/MatrixFilterSummary.helpers.js", + "app/ui/src/components/matrix/MatrixFilterSummary.jsx", + "app/ui/src/components/matrix/MatrixFilterWizard.helpers.js", + "app/ui/src/components/matrix/MatrixFilterWizard.jsx", + "app/ui/src/components/matrix/MatrixGroupRow.jsx", + "app/ui/src/components/matrix/MatrixGroupingCell.jsx", + "app/ui/src/components/matrix/MatrixGroupingRow.jsx", + "app/ui/src/components/matrix/MatrixLegend.jsx", + "app/ui/src/components/matrix/MatrixNamesRow.jsx", + "app/ui/src/components/matrix/MatrixScopePanel.jsx", + "app/ui/src/components/matrix/MatrixSubjectNameCell.jsx", + "app/ui/src/components/matrix/MatrixToolbar.jsx", + "app/ui/src/components/matrix/SortableMatrixBody.jsx", + "app/ui/src/components/matrix/accessPackageModel.js", + "app/ui/src/components/matrix/columnModel.js", + "app/ui/src/components/matrix/foldState.js", + "app/ui/src/components/matrix/matrixModel.js", + "app/ui/src/components/matrix/nestedRows.js", + "app/ui/src/components/matrix/sortUsers.js", + "app/ui/src/components/matrix/useHierarchyReset.js", + "app/ui/src/contexts/ThemeContext.jsx", + "app/ui/src/hooks/useContextTrees.js", + "app/ui/src/hooks/useDebouncedValue.js", + "app/ui/src/hooks/useDocsUrl.js", + "app/ui/src/hooks/useElapsedTimer.js", + "app/ui/src/hooks/useEntityPage.js", + "app/ui/src/hooks/useExpandableGraph.js", + "app/ui/src/hooks/useFeatures.js", + "app/ui/src/hooks/useFetch.js", + "app/ui/src/hooks/useMatrix.js", + "app/ui/src/hooks/useMatrixRowOrder.js", + "app/ui/src/hooks/useNestedGroupExpand.js", + "app/ui/src/hooks/usePermissions.js", + "app/ui/src/hooks/usePersistedState.js", + "app/ui/src/hooks/useRecentChanges.js", + "app/ui/src/hooks/useTheme.js", + "app/ui/src/hooks/useTimeline.js", + "app/ui/src/main.jsx", + "app/ui/src/pageRegistry.jsx", + "app/ui/src/utils/accessPackageStyles.js", + "app/ui/src/utils/attributeEntries.js", + "app/ui/src/utils/clipboard.js", + "app/ui/src/utils/colors.js", + "app/ui/src/utils/contextStyles.js", + "app/ui/src/utils/crawlerCredentials.js", + "app/ui/src/utils/docsUrl.js", + "app/ui/src/utils/excelHelpers.js", + "app/ui/src/utils/exportAccessPackagesToExcel.helpers.js", + "app/ui/src/utils/exportAccessPackagesToExcel.js", + "app/ui/src/utils/exportRollupToExcel.js", + "app/ui/src/utils/exportToExcel.helpers.js", + "app/ui/src/utils/exportToExcel.js", + "app/ui/src/utils/formatters.js", + "app/ui/src/utils/linkedMembers.js", + "app/ui/src/utils/navTabs.js", + "app/ui/src/utils/renderAttribute.jsx", + "app/ui/src/utils/resourceContexts.js", + "app/ui/src/utils/tabBadge.js", + "app/ui/src/utils/tierStyles.js" + ] +} diff --git a/.ci/psmutant.config.json b/.ci/psmutant.config.json index 807626f10..929e0addd 100644 --- a/.ci/psmutant.config.json +++ b/.ci/psmutant.config.json @@ -1,6 +1,10 @@ { - "_comment": "PSMutant mutation-testing config for IdentityAtlas's PowerShell crawler layer. Scope is the pure ConvertTo-* record-shapers PLUS the shared ingest/retry/batch library every crawler routes through. The shared layer was added deliberately: a score measured only over pure shapers reads as a suite-wide quality claim while describing the easiest code in the tree — the shapers scored ~95.8% while Invoke-CrawlerIngest.ps1 scored 39.7% the first time it was measured (weak assertions: `Should -Invoke -Times N` is an at-least check, so 'does not retry a 400' passed even when the mutant retried five times). After strengthening the retry/batch assertions it sits at 82.9%. The remaining survivors there are log-format constants and clamped arithmetic — equivalent mutants no assertion should pin. Enforced: the build fails when the blended score drops below break. Run: Install-Module PSMutant; Invoke-PSMutation -ConfigFile .ci/psmutant.config.json -SourceRoot .", - "sandboxSubtrees": ["tools", "test", "setup"], + "_comment": "PSMutant mutation-testing config for IdentityAtlas's PowerShell crawler layer. Scope is the pure ConvertTo-* record-shapers PLUS the shared ingest/retry/batch library every crawler routes through. The shared layer was added deliberately: a score measured only over pure shapers reads as a suite-wide quality claim while describing the easiest code in the tree \u2014 the shapers scored ~95.8% while Invoke-CrawlerIngest.ps1 scored 39.7% the first time it was measured (weak assertions: `Should -Invoke -Times N` is an at-least check, so 'does not retry a 400' passed even when the mutant retried five times). After strengthening the retry/batch assertions it sits at 82.9%. The remaining survivors there are log-format constants and clamped arithmetic \u2014 equivalent mutants no assertion should pin. Enforced: the build fails when the blended score drops below break. Run: Install-Module PSMutant; Invoke-PSMutation -ConfigFile .ci/psmutant.config.json -SourceRoot .", + "sandboxSubtrees": [ + "tools", + "test", + "setup" + ], "mutate": [ "tools/crawlers/entra-id/EntraIDCrawler.Transform.ps1", "tools/crawlers/entra-id/EntraIDCrawler.AppOwners.ps1", @@ -110,163 +114,374 @@ "tools/powershell-sdk/graph/Update-FGAccessTokenIfExpired.ps1", "tools/powershell-sdk/graph/Update-FGConfig.ps1", "tools/powershell-sdk/graph/Use-FGExistingAccessTokenString.ps1", - "tools/powershell-sdk/graph/Use-FGExistingMSALToken.ps1" + "tools/powershell-sdk/graph/Use-FGExistingMSALToken.ps1", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1" ], "tests": { - "tools/crawlers/entra-id/EntraIDCrawler.Transform.ps1": ["test/unit/EntraIDCrawlerTransform.Tests.ps1"], - "tools/crawlers/entra-id/EntraIDCrawler.AppOwners.ps1": ["test/unit/EntraIDCrawlerAppOwners.Tests.ps1", "test/unit/EntraIDCrawlerTransform.Tests.ps1"], - "tools/crawlers/entra-id/EntraIDCrawler.AppPermissions.ps1": ["test/unit/EntraIDCrawlerAppPermissions.Tests.ps1"], - "tools/crawlers/entra-id/EntraIDCrawler.PrincipalRelationships.ps1": ["test/unit/EntraIDCrawlerPrincipalRelationships.Tests.ps1"], - "tools/crawlers/shared/Get-CapabilityId.ps1": ["test/unit/CapabilityId.Tests.ps1"], - "tools/crawlers/shared/Invoke-CrawlerIngest.ps1": ["test/unit/CrawlerIngest.Tests.ps1", "test/unit/CrawlerIngestBatch.Tests.ps1"], - "tools/crawlers/entra-id/EntraIDCrawler.AppRoles.ps1": ["test/unit/EntraIDCrawlerTransform.Tests.ps1", "test/unit/EntraIDCrawlerAppOwners.Tests.ps1", "test/unit/EntraIDCrawlerAppPermissions.Tests.ps1"], - "tools/crawlers/entra-id/EntraIDCrawler.Functions.ps1": ["test/unit/EntraIDCrawlerFunctions.Tests.ps1"], - "tools/crawlers/azure-rm/AzureRMCrawler.Functions.ps1": ["test/unit/AzureRMCrawlerFunctions.Tests.ps1"], - "tools/crawlers/omada/OmadaCrawler.Functions.ps1": ["test/unit/OmadaCrawlerFunctions.Tests.ps1"], - "tools/crawlers/csv/CSVCrawler.Functions.ps1": ["test/unit/CSVCrawlerFunctions.Tests.ps1"], - "tools/crawlers/midpoint/MidpointCrawler.Functions.ps1": ["test/unit/MidpointCrawlerFunctions.Tests.ps1"], - "tools/crawlers/odata/Invoke-ODataAuth.ps1": ["test/unit/ODataLibrary.Tests.ps1"], - "tools/crawlers/odata/Invoke-ODataGetRequest.ps1": ["test/unit/ODataLibrary.Tests.ps1"], - "tools/crawlers/odata/Invoke-ODataPagedRequest.ps1": ["test/unit/ODataLibrary.Tests.ps1"], - "tools/crawlers/azure-rm/Get-AzureRMHelpers.ps1": ["test/unit/CrawlerHelpers.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGGetPage.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/crawlers/midpoint/Invoke-MidpointApi.ps1": ["test/unit/CrawlerHelpers.Tests.ps1"], - "tools/crawlers/azure-rm/Get-AzureRGHelpers.ps1": ["test/unit/CrawlerHelpers.Tests.ps1"], - "tools/crawlers/midpoint/MidpointCrawler.Transform.ps1": ["test/unit/MidpointCrawlerTransform.Tests.ps1"], - "tools/crawlers/omada/OmadaCrawler.Transform.ps1": ["test/unit/OmadaCrawlerTransform.Tests.ps1"], - "tools/crawlers/csv/CSVCrawler.Transform.ps1": ["test/unit/CSVCrawlerTransform.Tests.ps1"], - "tools/crawlers/azure-rm/AzureRMCrawler.Transform.ps1": ["test/unit/AzureRMCrawlerTransform.Tests.ps1"], - "tools/crawlers/azure-rm/AzureRMCrawler.Phases.ps1": ["test/unit/AzureRMCrawlerPhases.Tests.ps1"], - "tools/crawlers/csv/CSVCrawler.Phases.ps1": ["test/unit/CSVCrawlerPhases.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGAccessPackage.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGAccessPackagePolicy.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGAccessPackageResource.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGCatalog.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGGroup.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGGroupInCatalog.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGGroupMember.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGNotGroupMember.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Confirm-FGUser.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Convert-FGDistinguishedNameToOUPath.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Get-FGEntraPortalLink.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Get-FGServicePrincipalType.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Resolve-FGMemberObjectIds.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Test-FGDistinguishedName.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/helpers/Add-FGEntraCalculatedAttributes.ps1": ["test/unit/SdkHelpers.Tests.ps1"], - "tools/powershell-sdk/graph/Add-FGGroupMember.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Add-FGGroupToAccessPackage.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Add-FGGroupToCatalog.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Clear-FGSecureConfigValue.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Confirm-FGAccessTokenValidity.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAccessPackage.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAccessPackagesAssignments.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAccessPackagesPolicy.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAccessPackagesResource.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAccessToken.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAccessTokenDetail.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAccessTokenWithRefreshToken.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGApplication.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGApplicationExtensionProperty.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGAttributeMapping.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGCatalog.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGCatalogGroup.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGDevice.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGGroup.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGGroupEligibleMemberAll.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGGroupMember.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGGroupMemberAll.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGGroupMemberAllToFile.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGGroupTransitiveMemberAll.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGGroupTransitiveMemberAllToFile.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGObject.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGSecureConfigValue.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGServicePrincipal.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGServicePrincipalWithSync.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGSynchronizationJob.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGSynchronizationSchema.ps1": ["test/unit/SdkGovernance.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGUser.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGUserAccessPackagesAssignments.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGUserMail.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGUserMailFolder.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGUserManager.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Get-FGUserMemberOf.ps1": ["test/unit/SdkUserGroup.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGDeleteRequest.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGGetRequest.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGGetRequestStream.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGGetRequestToFile.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGPatchRequest.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGPostRequest.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGPutRequest.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Invoke-FGWriteRequest.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Merge-FGJsonArrayFile.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/New-FGAccessPackage.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/New-FGAccessPackagePolicy.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/New-FGCatalog.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/New-FGConnectedOrganization.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/New-FGGroup.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/New-FGServicePrincipalSecret.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Read-FGToken.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Remove-FGAccessPackage.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Remove-FGDevice.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Remove-FGGroupMember.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Remove-FGTrailingCommaFromJsonFile.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Save-FGToken.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Set-FGAccessPackage.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Set-FGAccessPackagePolicy.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Set-FGDevice.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Set-FGGroup.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Set-FGUser.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Test-FGConnection.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Test-FGSecureConfigValue.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Update-FGAccessTokenIfExpired.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Update-FGConfig.ps1": ["test/unit/SdkWrite.Tests.ps1"], - "tools/powershell-sdk/graph/Use-FGExistingAccessTokenString.ps1": ["test/unit/SdkRequests.Tests.ps1"], - "tools/powershell-sdk/graph/Use-FGExistingMSALToken.ps1": ["test/unit/SdkRequests.Tests.ps1"] + "tools/crawlers/entra-id/EntraIDCrawler.Transform.ps1": [ + "test/unit/EntraIDCrawlerTransform.Tests.ps1" + ], + "tools/crawlers/entra-id/EntraIDCrawler.AppOwners.ps1": [ + "test/unit/EntraIDCrawlerAppOwners.Tests.ps1", + "test/unit/EntraIDCrawlerTransform.Tests.ps1" + ], + "tools/crawlers/entra-id/EntraIDCrawler.AppPermissions.ps1": [ + "test/unit/EntraIDCrawlerAppPermissions.Tests.ps1" + ], + "tools/crawlers/entra-id/EntraIDCrawler.PrincipalRelationships.ps1": [ + "test/unit/EntraIDCrawlerPrincipalRelationships.Tests.ps1" + ], + "tools/crawlers/shared/Get-CapabilityId.ps1": [ + "test/unit/CapabilityId.Tests.ps1" + ], + "tools/crawlers/shared/Invoke-CrawlerIngest.ps1": [ + "test/unit/CrawlerIngest.Tests.ps1", + "test/unit/CrawlerIngestBatch.Tests.ps1" + ], + "tools/crawlers/entra-id/EntraIDCrawler.AppRoles.ps1": [ + "test/unit/EntraIDCrawlerTransform.Tests.ps1", + "test/unit/EntraIDCrawlerAppOwners.Tests.ps1", + "test/unit/EntraIDCrawlerAppPermissions.Tests.ps1" + ], + "tools/crawlers/entra-id/EntraIDCrawler.Functions.ps1": [ + "test/unit/EntraIDCrawlerFunctions.Tests.ps1" + ], + "tools/crawlers/azure-rm/AzureRMCrawler.Functions.ps1": [ + "test/unit/AzureRMCrawlerFunctions.Tests.ps1" + ], + "tools/crawlers/omada/OmadaCrawler.Functions.ps1": [ + "test/unit/OmadaCrawlerFunctions.Tests.ps1" + ], + "tools/crawlers/csv/CSVCrawler.Functions.ps1": [ + "test/unit/CSVCrawlerFunctions.Tests.ps1" + ], + "tools/crawlers/midpoint/MidpointCrawler.Functions.ps1": [ + "test/unit/MidpointCrawlerFunctions.Tests.ps1" + ], + "tools/crawlers/odata/Invoke-ODataAuth.ps1": [ + "test/unit/ODataLibrary.Tests.ps1" + ], + "tools/crawlers/odata/Invoke-ODataGetRequest.ps1": [ + "test/unit/ODataLibrary.Tests.ps1" + ], + "tools/crawlers/odata/Invoke-ODataPagedRequest.ps1": [ + "test/unit/ODataLibrary.Tests.ps1" + ], + "tools/crawlers/azure-rm/Get-AzureRMHelpers.ps1": [ + "test/unit/CrawlerHelpers.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGGetPage.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/crawlers/midpoint/Invoke-MidpointApi.ps1": [ + "test/unit/CrawlerHelpers.Tests.ps1" + ], + "tools/crawlers/azure-rm/Get-AzureRGHelpers.ps1": [ + "test/unit/CrawlerHelpers.Tests.ps1" + ], + "tools/crawlers/midpoint/MidpointCrawler.Transform.ps1": [ + "test/unit/MidpointCrawlerTransform.Tests.ps1" + ], + "tools/crawlers/omada/OmadaCrawler.Transform.ps1": [ + "test/unit/OmadaCrawlerTransform.Tests.ps1" + ], + "tools/crawlers/csv/CSVCrawler.Transform.ps1": [ + "test/unit/CSVCrawlerTransform.Tests.ps1" + ], + "tools/crawlers/azure-rm/AzureRMCrawler.Transform.ps1": [ + "test/unit/AzureRMCrawlerTransform.Tests.ps1" + ], + "tools/crawlers/azure-rm/AzureRMCrawler.Phases.ps1": [ + "test/unit/AzureRMCrawlerPhases.Tests.ps1" + ], + "tools/crawlers/csv/CSVCrawler.Phases.ps1": [ + "test/unit/CSVCrawlerPhases.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGAccessPackage.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGAccessPackagePolicy.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGAccessPackageResource.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGCatalog.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGGroup.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGGroupInCatalog.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGGroupMember.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGNotGroupMember.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Confirm-FGUser.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Convert-FGDistinguishedNameToOUPath.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Get-FGEntraPortalLink.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Get-FGServicePrincipalType.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Resolve-FGMemberObjectIds.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Test-FGDistinguishedName.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/helpers/Add-FGEntraCalculatedAttributes.ps1": [ + "test/unit/SdkHelpers.Tests.ps1" + ], + "tools/powershell-sdk/graph/Add-FGGroupMember.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Add-FGGroupToAccessPackage.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Add-FGGroupToCatalog.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Clear-FGSecureConfigValue.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Confirm-FGAccessTokenValidity.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAccessPackage.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAccessPackagesAssignments.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAccessPackagesPolicy.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAccessPackagesResource.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAccessToken.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAccessTokenDetail.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAccessTokenWithRefreshToken.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGApplication.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGApplicationExtensionProperty.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGAttributeMapping.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGCatalog.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGCatalogGroup.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGDevice.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGGroup.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGGroupEligibleMemberAll.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGGroupMember.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGGroupMemberAll.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGGroupMemberAllToFile.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGGroupTransitiveMemberAll.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGGroupTransitiveMemberAllToFile.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGObject.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGSecureConfigValue.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGServicePrincipal.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGServicePrincipalWithSync.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGSynchronizationJob.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGSynchronizationSchema.ps1": [ + "test/unit/SdkGovernance.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGUser.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGUserAccessPackagesAssignments.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGUserMail.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGUserMailFolder.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGUserManager.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Get-FGUserMemberOf.ps1": [ + "test/unit/SdkUserGroup.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGDeleteRequest.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGGetRequest.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGGetRequestStream.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGGetRequestToFile.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGPatchRequest.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGPostRequest.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGPutRequest.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Invoke-FGWriteRequest.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Merge-FGJsonArrayFile.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/New-FGAccessPackage.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/New-FGAccessPackagePolicy.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/New-FGCatalog.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/New-FGConnectedOrganization.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/New-FGGroup.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/New-FGServicePrincipalSecret.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Read-FGToken.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Remove-FGAccessPackage.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Remove-FGDevice.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Remove-FGGroupMember.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Remove-FGTrailingCommaFromJsonFile.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Save-FGToken.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Set-FGAccessPackage.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Set-FGAccessPackagePolicy.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Set-FGDevice.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Set-FGGroup.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Set-FGUser.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Test-FGConnection.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Test-FGSecureConfigValue.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Update-FGAccessTokenIfExpired.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Update-FGConfig.ps1": [ + "test/unit/SdkWrite.Tests.ps1" + ], + "tools/powershell-sdk/graph/Use-FGExistingAccessTokenString.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/powershell-sdk/graph/Use-FGExistingMSALToken.ps1": [ + "test/unit/SdkRequests.Tests.ps1" + ], + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1": [ + "test/unit/EntraIDCrawlerPhases.Tests.ps1" + ], + "tools/crawlers/omada/OmadaCrawler.Phases.ps1": [ + "test/unit/OmadaCrawlerPhases.Tests.ps1" + ], + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1": [ + "test/unit/MidpointCrawlerPhases.Tests.ps1" + ] }, - "_exclusionsComment": "Eligible files deliberately NOT mutation-tested. Each key is a repo-relative path; the value is the reason. The scope-completeness guard (test/unit/PSMutationScope.Tests.ps1, #684) fails an eligible file that is in neither `mutate` nor here, so anything listed is a written decision rather than an oversight. NOTE: this map covers tools/crawlers/. The other two PowerShell roots — tools/powershell-sdk/ and tools/riskscoring/ — are eligible too, and the ones not yet triaged are grandfathered in .ci/psmutant-scope-baseline.json, a list that may only shrink (102 at introduction, 88 after the helpers layer was measured). A new file in those roots still has to land in `mutate` or `exclusions` here; it cannot join the backlog silently. First finding out of that backlog: the SDK is NOT untested code — tools/powershell-sdk/ + tools/riskscoring/ sit at 95.7% line coverage — it was code with no mutation evidence at all, and the helpers layer scored 85.5% (94/110) the first time it was measured.", + "_exclusionsComment": "Eligible files deliberately NOT mutation-tested. Each key is a repo-relative path; the value is the reason. The scope-completeness guard (test/unit/PSMutationScope.Tests.ps1, #684) fails an eligible file that is in neither `mutate` nor here, so anything listed is a written decision rather than an oversight. NOTE: this map covers tools/crawlers/. The other two PowerShell roots \u2014 tools/powershell-sdk/ and tools/riskscoring/ \u2014 are eligible too, and the ones not yet triaged are grandfathered in .ci/psmutant-scope-baseline.json, a list that may only shrink (102 at introduction, 88 after the helpers layer was measured). A new file in those roots still has to land in `mutate` or `exclusions` here; it cannot join the backlog silently. First finding out of that backlog: the SDK is NOT untested code \u2014 tools/powershell-sdk/ + tools/riskscoring/ sit at 95.7% line coverage \u2014 it was code with no mutation evidence at all, and the helpers layer scored 85.5% (94/110) the first time it was measured.", "exclusions": { "tools/powershell-sdk/graph/Get-FGAccessTokenInteractive.ps1": "MEASURED: produces ZERO mutants, because no suite reaches it -- it is the only file in tools/powershell-sdk/ with no test coverage at all, so coveredLinesOnly filters every candidate out and mutating it would report a vacuous 100%. That is not an oversight to fix with a mock: it is the OAuth device-code flow. It prints a code for a human to type into a browser and then polls Microsoft in a `while (IsNullOrEmpty($TokenRequest.access_token))` loop until they finish. There is no headless path through it, and a test that mocked the poll would assert only that the mock was called. Revisit if the flow is ever split so the response-shaping half can be exercised without the wait.", - - "tools/crawlers/omada/Get-OmadaHelpers.ps1": "Produces ZERO mutants under the enabled operator set, so mutating it would report a vacuous 100%. It is five reference-value fallback chains (`if ($Ref.Value) { return [string]$Ref.Value }` repeated) with no arithmetic, no numeric/boolean literals and no negations for BinaryOperator/BooleanLiteral/NumberLiteral/NegationRemoval to bite on — the precedence order itself is only reachable by a statement-deletion or condition-forcing operator PSMutant does not currently offer. Revisit if such an operator is added; its behaviour is covered by OmadaCrawlerFunctions.Tests.ps1 in the meantime.", - - - "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1": "MEASURED at 39.6% (101/255) against EntraIDCrawlerPhases.Tests.ps1. CORRECTION: this entry previously claimed the Phases files are monoliths with functions trapped in a top-level Main body and therefore untestable without a decomposition first. That is false, and it was written without checking. It describes the Start-*Crawler.ps1 entry points, not these — the extraction already happened, and the layer is 158 named functions across five files with 263 existing tests. It is measurable today; it is simply weak. Being worked file by file: AzureRMCrawler.Phases.ps1 went 71.2% -> 84.8% and CSVCrawler.Phases.ps1 went 47.8% -> 82.1%; both are now in `mutate` above. The shared gap across the layer is that the existing tests count what a phase produced (`.Count | Should -Be n`, `-BeGreaterThan 0`) instead of asserting what is IN it, so an inverted guard that changes an edge's direction or duplicates a record leaves the counts intact. The fix is per-file test-writing, not a config change.", - "tools/crawlers/omada/OmadaCrawler.Phases.ps1": "MEASURED at 57.8% (93/161) against OmadaCrawlerPhases.Tests.ps1 — see EntraIDCrawler.Phases.ps1 above for why the layer is deferred.", - "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1": "MEASURED at 55.9% (71/127) against MidpointCrawlerPhases.Tests.ps1 — see EntraIDCrawler.Phases.ps1 above.", + "tools/crawlers/omada/Get-OmadaHelpers.ps1": "Produces ZERO mutants under the enabled operator set, so mutating it would report a vacuous 100%. It is five reference-value fallback chains (`if ($Ref.Value) { return [string]$Ref.Value }` repeated) with no arithmetic, no numeric/boolean literals and no negations for BinaryOperator/BooleanLiteral/NumberLiteral/NegationRemoval to bite on \u2014 the precedence order itself is only reachable by a statement-deletion or condition-forcing operator PSMutant does not currently offer. Revisit if such an operator is added; its behaviour is covered by OmadaCrawlerFunctions.Tests.ps1 in the meantime.", "tools/crawlers/entra-id/EntraIDCrawler.Orchestration.ps1": "MEASURED: produces ZERO mutants under the enabled operator set, so mutating it would report a vacuous 100% \u2014 same category as Get-OmadaHelpers.ps1. It is 61 lines: one function whose body is five `if ($SyncX) { ... }` phase guards. Those guards ARE decisions, but they are bare boolean variable checks with no comparison operators, numeric/boolean literals or negations for BinaryOperator/BooleanLiteral/NumberLiteral/NegationRemoval to act on. (An earlier version of this entry said it 'owns no decision logic of its own' and deferred it alongside the Phases layer. That was reasoned, not measured, and it overstated: the guards are decisions, they are simply not mutatable by these operators. Which phases run on a given config is worth a behavioural test in EntraIDCrawlerPhases.Tests.ps1 regardless of what mutation testing can see.)", - "tools/riskscoring/Export-FGRiskClassifiers.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Export-FGRiskProfile.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Get-FGCorrelationRuleset.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Get-FGRiskClassifiers.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Get-FGRiskProfile.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Import-FGRiskClassifiers.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Import-FGRiskProfile.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Invoke-FGAccountCorrelation.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Invoke-FGLLMRequest.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Invoke-FGRiskScoring.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/New-FGCorrelationRuleset.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/New-FGRiskClassifiers.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/New-FGRiskProfile.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Save-FGCorrelationRuleset.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Save-FGResourceClusters.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Save-FGRiskClassifiers.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", - "tools/riskscoring/Save-FGRiskProfile.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage." }, "_equivalentsComment": "Mutants excluded from the mutation DENOMINATOR, each with a reason. PSMutant fails the build if a declared mutant is ever killed, or stops existing -- so a declaration is a claim that gets checked, not a mute button. Everything here is a progress-bar percentage; nothing with functional effect is listed. Read the individual reasons before adding to this map.", @@ -286,8 +501,7 @@ "tools/crawlers/csv/CSVCrawler.Phases.ps1:236:72 -> 73": "Progress-bar checkpoint for the IdentityMembers.csv phase, nudged by one point. NOT the usual 'cannot change behaviour' case, and the distinction matters: this IS observable -- it is written to the job progress the Crawlers UI renders -- so a test COULD pin it. The claim being made is narrower and is a judgement, not a proof: the exact number carries no meaning on its own. What the UI needs is that progress rises through the run and ends at 100, and both of those ARE tested (Complete-*Run asserts every value is a percentage and that the last is exactly 100). Pinning the intermediate values would add a change-detector that fails whenever anyone retunes the bar, while catching no defect a user could notice. Reviewers: this reason is the only thing standing behind these entries. Unlike a stale declaration, a WRONG one of this kind cannot be caught automatically -- no test will ever kill these to prove the claim false -- so weigh it, do not rubber-stamp it, and do not extend this pattern to a constant with functional effect. Two that looked like this were NOT declared and were tested instead: the 3000-record certification batch size, and the 300s refresh-views timeout.", "tools/crawlers/csv/CSVCrawler.Phases.ps1:257:78 -> 79": "Progress-bar checkpoint for the Certifications.csv phase, nudged by one point. NOT the usual 'cannot change behaviour' case, and the distinction matters: this IS observable -- it is written to the job progress the Crawlers UI renders -- so a test COULD pin it. The claim being made is narrower and is a judgement, not a proof: the exact number carries no meaning on its own. What the UI needs is that progress rises through the run and ends at 100, and both of those ARE tested (Complete-*Run asserts every value is a percentage and that the last is exactly 100). Pinning the intermediate values would add a change-detector that fails whenever anyone retunes the bar, while catching no defect a user could notice. Reviewers: this reason is the only thing standing behind these entries. Unlike a stale declaration, a WRONG one of this kind cannot be caught automatically -- no test will ever kill these to prove the claim false -- so weigh it, do not rubber-stamp it, and do not extend this pattern to a constant with functional effect. Two that looked like this were NOT declared and were tested instead: the 3000-record certification batch size, and the 300s refresh-views timeout.", "tools/crawlers/csv/CSVCrawler.Phases.ps1:325:85 -> 86": "Progress-bar checkpoint for the BusinessRole classification phase, nudged by one point. NOT the usual 'cannot change behaviour' case, and the distinction matters: this IS observable -- it is written to the job progress the Crawlers UI renders -- so a test COULD pin it. The claim being made is narrower and is a judgement, not a proof: the exact number carries no meaning on its own. What the UI needs is that progress rises through the run and ends at 100, and both of those ARE tested (Complete-*Run asserts every value is a percentage and that the last is exactly 100). Pinning the intermediate values would add a change-detector that fails whenever anyone retunes the bar, while catching no defect a user could notice. Reviewers: this reason is the only thing standing behind these entries. Unlike a stale declaration, a WRONG one of this kind cannot be caught automatically -- no test will ever kill these to prove the claim false -- so weigh it, do not rubber-stamp it, and do not extend this pattern to a constant with functional effect. Two that looked like this were NOT declared and were tested instead: the 3000-record certification batch size, and the 300s refresh-views timeout.", - "tools/crawlers/csv/CSVCrawler.Phases.ps1:331:88 -> 89": "Progress-bar checkpoint for the view refresh phase, nudged by one point. NOT the usual 'cannot change behaviour' case, and the distinction matters: this IS observable -- it is written to the job progress the Crawlers UI renders -- so a test COULD pin it. The claim being made is narrower and is a judgement, not a proof: the exact number carries no meaning on its own. What the UI needs is that progress rises through the run and ends at 100, and both of those ARE tested (Complete-*Run asserts every value is a percentage and that the last is exactly 100). Pinning the intermediate values would add a change-detector that fails whenever anyone retunes the bar, while catching no defect a user could notice. Reviewers: this reason is the only thing standing behind these entries. Unlike a stale declaration, a WRONG one of this kind cannot be caught automatically -- no test will ever kill these to prove the claim false -- so weigh it, do not rubber-stamp it, and do not extend this pattern to a constant with functional effect. Two that looked like this were NOT declared and were tested instead: the 3000-record certification batch size, and the 300s refresh-views timeout." -, + "tools/crawlers/csv/CSVCrawler.Phases.ps1:331:88 -> 89": "Progress-bar checkpoint for the view refresh phase, nudged by one point. NOT the usual 'cannot change behaviour' case, and the distinction matters: this IS observable -- it is written to the job progress the Crawlers UI renders -- so a test COULD pin it. The claim being made is narrower and is a judgement, not a proof: the exact number carries no meaning on its own. What the UI needs is that progress rises through the run and ends at 100, and both of those ARE tested (Complete-*Run asserts every value is a percentage and that the last is exactly 100). Pinning the intermediate values would add a change-detector that fails whenever anyone retunes the bar, while catching no defect a user could notice. Reviewers: this reason is the only thing standing behind these entries. Unlike a stale declaration, a WRONG one of this kind cannot be caught automatically -- no test will ever kill these to prove the claim false -- so weigh it, do not rubber-stamp it, and do not extend this pattern to a constant with functional effect. Two that looked like this were NOT declared and were tested instead: the 3000-record certification batch size, and the 300s refresh-views timeout.", "tools/powershell-sdk/graph/Add-FGGroupToAccessPackage.ps1:33:45 -> 46": "Start-Sleep -s 45 -> 46. A fixed pause waiting for Microsoft Graph to become eventually consistent before the next call. Observable in principle -- a test could pin the argument -- but the exact number is a guess about someone else's service, not a contract: one second either way changes nothing anyone can detect. What matters is THAT it waits, and that IS asserted (Should -Invoke Start-Sleep) in the Add-FGGroupTo* tests.", "tools/powershell-sdk/graph/Add-FGGroupToCatalog.ps1:24:45 -> 46": "Start-Sleep -s 45 -> 46. A fixed pause waiting for Microsoft Graph to become eventually consistent before the next call. Observable in principle -- a test could pin the argument -- but the exact number is a guess about someone else's service, not a contract: one second either way changes nothing anyone can detect. What matters is THAT it waits, and that IS asserted (Should -Invoke Start-Sleep) in the Add-FGGroupTo* tests.", "tools/powershell-sdk/graph/Clear-FGSecureConfigValue.ps1:84:10 -> 11": "ConvertTo-Json -Depth 10 -> 11. The payloads this serialises are far shallower than 10, so the two depths emit byte-identical JSON. This is the classic equivalent mutant: no test can separate N from N+1 without a fixture built for no purpose except to out-nest the serializer, which would assert nothing about the request being sent.", @@ -322,8 +536,17 @@ "tools/powershell-sdk/graph/Update-FGConfig.ps1:8:120 -> 121": "Progress/rate display arithmetic (120 -> 121). Feeds a Write-Progress or Write-Host counter and nothing else -- no caller, no file, no request. Same class as the crawler progress percentages declared above, and the same caveat applies: this IS observable, so the claim is a judgement about worth, not a proof of equivalence." }, "coveredLinesOnly": true, - "operators": ["BinaryOperator", "BooleanLiteral", "NumberLiteral", "NegationRemoval"], - "_thresholdsComment": "break was 90 when the scope was pure shapers only (which sat at ~95.8%). With the non-pure ingest layer included the blended score is 90.7% (283/312), leaving 0.7 points of headroom — one new survivor would fail the build, and the cheapest way to make that failure go away would be to drop the ingest layer back out of `mutate`. That is precisely the behaviour this scope widening exists to stop, so break is set to 85: still far above the `low` band, and a genuine regression (a whole file's assertions weakening) moves the blended score by much more than 5 points. Raise it as survivors are killed; never raise it by narrowing `mutate`.", - "thresholds": { "high": 90, "low": 70, "break": 85 }, + "operators": [ + "BinaryOperator", + "BooleanLiteral", + "NumberLiteral", + "NegationRemoval" + ], + "_thresholdsComment": "break was 90 when the scope was pure shapers only (which sat at ~95.8%). With the non-pure ingest layer included the blended score is 90.7% (283/312), leaving 0.7 points of headroom \u2014 one new survivor would fail the build, and the cheapest way to make that failure go away would be to drop the ingest layer back out of `mutate`. That is precisely the behaviour this scope widening exists to stop, so break is set to 85: still far above the `low` band, and a genuine regression (a whole file's assertions weakening) moves the blended score by much more than 5 points. Raise it as survivors are killed; never raise it by narrowing `mutate`.", + "thresholds": { + "high": 90, + "low": 70, + "break": 85 + }, "reportPath": "reports/ps-mutation.json" -} +} \ No newline at end of file diff --git a/app/api/src/effectiveAccess/engine.js b/app/api/src/effectiveAccess/engine.js index b3e08986d..083279677 100644 --- a/app/api/src/effectiveAccess/engine.js +++ b/app/api/src/effectiveAccess/engine.js @@ -19,6 +19,7 @@ import { groupForNodeGrants, emitNodeRows, } from './engine.helpers.js'; +import { createLru } from './lru.js'; export const DEFAULTS = { maxDepth: 50, @@ -35,33 +36,6 @@ export const DEFAULTS = { export const GROUP_RESOURCE_TYPES = ['Group']; // ── Minimal count-bounded LRU ──────────────────────────────────────────────── -// P1 placeholder for the `lru-cache` package (spec D3/D8 prescribe a byte-bounded cache); the -// correctness-relevant behavior — keying on dataVersion so a completed sync invalidates every -// entry — is identical. Swap the implementation when the dependency is wired; callers don't change. -function createLru(max) { - const map = new Map(); // insertion-ordered → front = oldest - return { - get(key) { - if (!map.has(key)) return undefined; - const v = map.get(key); - map.delete(key); - map.set(key, v); // move to most-recent - return v; - }, - set(key, v) { - if (map.has(key)) map.delete(key); - map.set(key, v); - while (map.size > max) map.delete(map.keys().next().value); - }, - get size() { - return map.size; - }, - clear() { - map.clear(); - }, - }; -} - const cache = createLru(DEFAULTS.cacheMaxEntries); // Exposed for tests / admin — drop all cached results (e.g. after a manual data change). diff --git a/app/api/src/effectiveAccess/engineContainment.test.js b/app/api/src/effectiveAccess/engineContainment.test.js index 587559c69..40bcd67df 100644 --- a/app/api/src/effectiveAccess/engineContainment.test.js +++ b/app/api/src/effectiveAccess/engineContainment.test.js @@ -70,6 +70,44 @@ describe('getAncestorNodes', () => { }); }); +describe('getAncestorNodes - the caps that stop the walk', () => { + // Truncation is how an access answer comes back INCOMPLETE, and nothing exercised either + // cap. Both are off-by-one sensitive and both fail quietly: a walk that stops one level + // early simply reports less access than the principal has, with no error anywhere. The + // fixture chain is vm -> rg -> sub, so depth 1 and a node budget of 2 both land exactly on + // the boundary -- which is the only place `>=` and `>` disagree. + it('stops at maxDepth, admitting exactly that many levels', async () => { + const { depthByNode, truncated } = await getAncestorNodes('vm', { maxDepth: 1 }); + expect(depthByNode.get('vm')).toBe(0); + expect(depthByNode.get('rg')).toBe(1); + expect(depthByNode.has('sub')).toBe(false); // one level further would need depth 2 + expect(truncated).toBeTruthy(); + }); + + it('does not truncate when the tree fits inside maxDepth', async () => { + // The paired case: without it, "always truncate" passes the test above. + const { depthByNode, truncated } = await getAncestorNodes('vm', { maxDepth: 9 }); + expect(depthByNode.get('sub')).toBe(2); + expect(truncated).toBeFalsy(); + }); + + it('stops at maxNodesPerExpansion, counting the start node', async () => { + // The budget counts nodes already admitted, and vm is admitted before the walk begins -- + // so a budget of 2 leaves room for exactly one more. + const { depthByNode, truncated } = await getAncestorNodes('vm', { maxNodesPerExpansion: 2 }); + expect(depthByNode.has('rg')).toBe(true); + expect(depthByNode.has('sub')).toBe(false); + expect(depthByNode.size).toBe(2); + expect(truncated).toBeTruthy(); + }); + + it('does not truncate when the tree fits inside the node budget', async () => { + const { depthByNode, truncated } = await getAncestorNodes('vm', { maxNodesPerExpansion: 50 }); + expect(depthByNode.size).toBe(3); + expect(truncated).toBeFalsy(); + }); +}); + describe('effectiveAccessAtNode', () => { it('inherits ancestor grants as Indirect, keeps focus-node grants Direct, drops self-scope-at-ancestor', async () => { const r = await effectiveAccessAtNode('vm', 'u1'); diff --git a/app/api/src/effectiveAccess/engineObservability.test.js b/app/api/src/effectiveAccess/engineObservability.test.js new file mode 100644 index 000000000..d033d3687 --- /dev/null +++ b/app/api/src/effectiveAccess/engineObservability.test.js @@ -0,0 +1,159 @@ +// The parts of the engine nothing had reached: the cache/telemetry line every resolve emits, +// the group-vs-self distinction behind the Direct badge, DAG de-duplication, and the +// multi-node entrypoint's early exit and truncation roll-up. +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import { getSyncVersion } from '../lib/syncVersion.js'; + +// No inline factory: mocking the module with no second argument picks up the shared manual +// mock at src/db/__mocks__/connection.js. src/db/connectionMock.test.js ratchets the number +// of inline mock factories for that module DOWNWARD, so adding one here would push the count +// up and fail the gate. (It matches on file TEXT, so even quoting such a call in a comment +// counts -- which is why this note describes the pattern instead of showing it.) +vi.mock('../db/connection.js'); +vi.mock('../lib/syncVersion.js', () => ({ getSyncVersion: vi.fn() })); +import * as db from '../db/connection.js'; + +const { + effectiveAccess, effectiveAccessForNodes, getAncestorNodes, clearCache, +} = await import('./engine.js'); + +// u1 is a member of group g1; the Contributor grant is held by the GROUP, not by u1. +const membership = { u1: ['g1'] }; +const grants = [{ cap: 'Contributor', target: 'sub', holder: 'g1', effect: 'allow', scope: 'selfAndDescendants' }]; + +function wire({ contains = {}, rows = grants } = {}) { + db.query.mockImplementation((sql, params) => { + if (sql.includes('ResourceRelationships')) { + const parents = new Set(); + for (const c of params[0]) for (const e of contains[c] || []) parents.add(e); + return Promise.resolve({ rows: [...parents].map((parent) => ({ parent })) }); + } + if (sql.includes('resourceType')) { + const gids = new Set(); + for (const p of params[0]) for (const g of membership[p] || []) gids.add(g); + return Promise.resolve({ rows: [...gids].map((gid) => ({ gid })) }); + } + if (sql.includes('ResourceAssignments')) { + // resolveForPrincipalOnResource: grants ON this resource, held by anyone in the + // holder set (the principal plus their groups). Shape is {holder, effect}. + return Promise.resolve({ + rows: rows + .filter((g) => g.target === params[0] && params[1].includes(g.holder)) + .map((g) => ({ holder: g.holder, effect: g.effect })), + }); + } + if (sql.includes('displayName')) return Promise.resolve({ rows: [] }); + return Promise.resolve({ rows: [] }); + }); +} + +beforeEach(() => { + vi.clearAllMocks(); + clearCache(); + wire(); + getSyncVersion.mockResolvedValue(1); +}); + +describe('effectiveAccess - the observability line', () => { + let logged; + beforeEach(() => { + logged = []; + vi.spyOn(console, 'log').mockImplementation((s) => logged.push(JSON.parse(s))); + }); + afterEach(() => vi.restoreAllMocks()); + + it('reports cacheHit false on the first call and true on the second', async () => { + // One line per resolve is the documented contract (spec 19), and cacheHit is the only + // signal that the cache is doing anything at all. Hard-code either value and the line + // still looks perfectly well-formed. + await effectiveAccess('sub', 'u1'); + await effectiveAccess('sub', 'u1'); + + expect(logged).toHaveLength(2); + expect(logged[0].cacheHit).toBe(false); + expect(logged[1].cacheHit).toBe(true); + expect(logged[0].event).toBe('effective-access-resolve'); + expect(logged[0].focusNode).toBe('sub'); + expect(logged[0].principalId).toBe('u1'); + }); + + it('reports truncated as a boolean, not the truncation object', async () => { + // `!!result.truncated` collapses an object to true. Dropping one `!` inverts it; dropping + // both leaks the object into a field consumers read as a flag. + await effectiveAccess('sub', 'u1'); + expect(typeof logged[0].truncated).toBe('boolean'); + expect(logged[0].truncated).toBe(false); + }); + + it('reports a non-negative duration', async () => { + // Date.now() - started. Read as +, this is ~1.7e12 rather than single digits. + await effectiveAccess('sub', 'u1'); + expect(logged[0].durationMs).toBeGreaterThanOrEqual(0); + expect(logged[0].durationMs).toBeLessThan(60_000); + }); +}); + +describe('resolveForPrincipalOnResource - who actually holds the grant', () => { + it('marks a grant held by the principals GROUP as not explicit', async () => { + // explicit drives the Direct badge. Hard-coded true, a grant a user only has through + // group membership is reported as held directly by them -- the badge says Direct and the + // group it really came from disappears from the answer. + const r = await effectiveAccess('sub', 'u1'); + expect(r.effective).toBe('allow'); + expect(r.decisiveAce.explicit).toBe(false); + expect(r.decisiveAce.viaGroupId).toBe('g1'); + }); + + it('marks a grant held by the principal themself as explicit', async () => { + wire({ rows: [{ cap: 'Contributor', target: 'sub', holder: 'u1', effect: 'allow', scope: 'selfAndDescendants' }] }); + const r = await effectiveAccess('sub', 'u1'); + expect(r.decisiveAce.explicit).toBe(true); + expect(r.decisiveAce.viaGroupId).toBeNull(); + }); +}); + +describe('getAncestorNodes - DAG de-duplication', () => { + it('admits a node reached by two different paths exactly once, at its shortest depth', async () => { + // Diamond: vm -> a and vm -> b, both -> top. Without the "already admitted" check top is + // re-admitted at the later depth, and on a cyclic graph the walk would not terminate. + wire({ contains: { vm: ['a', 'b'], a: ['top'], b: ['top'] } }); + const { depthByNode } = await getAncestorNodes('vm'); + expect(depthByNode.get('vm')).toBe(0); + expect(depthByNode.get('a')).toBe(1); + expect(depthByNode.get('b')).toBe(1); + expect(depthByNode.get('top')).toBe(2); + expect(depthByNode.size).toBe(4); + }); +}); + +describe('effectiveAccessForNodes', () => { + it('returns empty for no focus nodes without touching the database', async () => { + const r = await effectiveAccessForNodes([]); + expect(r).toEqual({ rows: [], truncated: null }); + expect(db.query).not.toHaveBeenCalled(); + }); + + it('treats a null focus list the same way', async () => { + expect(await effectiveAccessForNodes(null)).toEqual({ rows: [], truncated: null }); + }); + + it('de-duplicates repeated focus nodes', async () => { + await effectiveAccessForNodes(['sub', 'sub']); + const nameCall = db.query.mock.calls.find(([sql]) => sql.includes('displayName')); + expect(nameCall[1][0]).toEqual(['sub']); + }); + + it('reports no truncation when nothing truncated', async () => { + // The paired case below is what stops "always truncated" passing. + const r = await effectiveAccessForNodes(['sub']); + expect(r.truncated).toBeNull(); + }); + + it('rolls a single truncated node up into the overall result', async () => { + // One node hitting a cap makes the WHOLE answer incomplete; reporting null there would + // present a partial result as a complete one. + wire({ contains: { sub: ['p1'], p1: ['p2'] } }); + const r = await effectiveAccessForNodes(['sub'], { maxDepth: 1 }); + expect(r.truncated).toEqual({ ancestors: true }); + }); +}); diff --git a/app/api/src/effectiveAccess/lru.js b/app/api/src/effectiveAccess/lru.js new file mode 100644 index 000000000..1b05968be --- /dev/null +++ b/app/api/src/effectiveAccess/lru.js @@ -0,0 +1,35 @@ +// A small least-recently-used cache for resolved effective-access results. +// +// Extracted from engine.js so it can be tested and measured directly. Inside the engine it +// was reachable only through resolve(), which needs a database and enough distinct keys to +// force an eviction -- so nothing exercised the recency or eviction behaviour at all, and a +// broken cache is silent in both directions: it either serves an access decision that is no +// longer true, or never evicts and grows without bound. +// +// P1 placeholder for the `lru-cache` package (spec D3/D8 prescribe a byte-bounded cache); the +// correctness-relevant behaviour -- keying on dataVersion so a completed sync invalidates +// every entry -- is identical. Swap the implementation when the dependency is wired; callers +// don't change. +export function createLru(max) { + const map = new Map(); // insertion-ordered → front = oldest + return { + get(key) { + if (!map.has(key)) return undefined; + const v = map.get(key); + map.delete(key); + map.set(key, v); // move to most-recent + return v; + }, + set(key, v) { + if (map.has(key)) map.delete(key); + map.set(key, v); + while (map.size > max) map.delete(map.keys().next().value); + }, + get size() { + return map.size; + }, + clear() { + map.clear(); + }, + }; +} diff --git a/app/api/src/effectiveAccess/lru.test.js b/app/api/src/effectiveAccess/lru.test.js new file mode 100644 index 000000000..546c631d7 --- /dev/null +++ b/app/api/src/effectiveAccess/lru.test.js @@ -0,0 +1,114 @@ +// The cache decides whether a caller sees a freshly computed access decision or a stored +// one. Both failure directions are silent: evict too eagerly and it is merely slow, evict +// too late (or never) and it serves access that is no longer true, or grows without bound. +// None of this was exercised while the cache lived inside engine.js, because reaching it +// meant going through resolve() with a database and enough distinct keys to force eviction. +import { describe, it, expect } from 'vitest'; +import { createLru } from './lru.js'; + +describe('createLru - basics', () => { + it('stores and returns a value', () => { + const c = createLru(3); + c.set('a', 1); + expect(c.get('a')).toBe(1); + expect(c.size).toBe(1); + }); + + it('returns undefined for a key it does not hold', () => { + // The miss path must not throw or return a stale/adjacent value; the caller treats + // undefined as "recompute". + expect(createLru(3).get('nope')).toBeUndefined(); + }); + + it('overwrites a key in place rather than storing it twice', () => { + const c = createLru(3); + c.set('a', 1); + c.set('a', 2); + expect(c.get('a')).toBe(2); + expect(c.size).toBe(1); + }); + + it('clear() empties it', () => { + const c = createLru(3); + c.set('a', 1); + c.set('b', 2); + c.clear(); + expect(c.size).toBe(0); + expect(c.get('a')).toBeUndefined(); + }); +}); + +describe('createLru - eviction', () => { + it('evicts the OLDEST entry once it is over capacity, and only then', () => { + // Exactly at capacity nothing may be dropped -- the boundary between `size > max` and + // `size >= max` is one whole entry of cache, and the second reading throws away a live + // result on every single write. + const c = createLru(2); + c.set('a', 1); + c.set('b', 2); + expect(c.size).toBe(2); + expect(c.get('a')).toBe(1); // still there at exactly max + + c.set('c', 3); // now over capacity + expect(c.size).toBe(2); + }); + + it('evicts by insertion order when nothing has been read', () => { + const c = createLru(2); + c.set('a', 1); + c.set('b', 2); + c.set('c', 3); + expect(c.get('a')).toBeUndefined(); // oldest, dropped + expect(c.get('b')).toBe(2); + expect(c.get('c')).toBe(3); + }); + + it('never grows past capacity, however many writes arrive', () => { + const c = createLru(3); + for (let i = 0; i < 50; i++) c.set(`k${i}`, i); + expect(c.size).toBe(3); + expect(c.get('k49')).toBe(49); + expect(c.get('k0')).toBeUndefined(); + }); +}); + +describe('createLru - recency', () => { + it('a READ protects an entry from the next eviction', () => { + // This is the whole point of the L in LRU. Without the re-insert on get(), 'a' is still + // the oldest by insertion order and gets dropped despite being the most recently used + // -- so the hottest access decision is the one thrown away. + const c = createLru(2); + c.set('a', 1); + c.set('b', 2); + expect(c.get('a')).toBe(1); // 'a' becomes most-recent, 'b' is now oldest + c.set('c', 3); + + expect(c.get('a')).toBe(1); // survived because it was read + expect(c.get('b')).toBeUndefined(); // evicted instead + expect(c.get('c')).toBe(3); + }); + + it('a re-WRITE also refreshes recency', () => { + const c = createLru(2); + c.set('a', 1); + c.set('b', 2); + c.set('a', 11); // refreshes 'a' + c.set('c', 3); + + expect(c.get('a')).toBe(11); + expect(c.get('b')).toBeUndefined(); + }); + + it('a MISSED read does not disturb the eviction order', () => { + // get() on an absent key must return early: if it fell through to the re-insert it + // would create an entry, or reorder the ones already there. + const c = createLru(2); + c.set('a', 1); + c.set('b', 2); + expect(c.get('ghost')).toBeUndefined(); + expect(c.size).toBe(2); + c.set('c', 3); + expect(c.get('a')).toBeUndefined(); // still the oldest; the miss changed nothing + expect(c.get('b')).toBe(2); + }); +}); diff --git a/app/api/src/effectiveAccess/policies.test.js b/app/api/src/effectiveAccess/policies.test.js index aded8f69d..e43aeee9a 100644 --- a/app/api/src/effectiveAccess/policies.test.js +++ b/app/api/src/effectiveAccess/policies.test.js @@ -61,6 +61,65 @@ describe('AdditiveAllow.resolve', () => { }); }); +describe('AdditiveAllow.resolve - decisive pick, with the winner listed FIRST', () => { + // The tests above all list the expected winner LAST, so "always keep the ace we just + // looked at" produces the same answer as "pick the best one" -- the reduce could ignore + // its accumulator entirely and still pass. These put the winner first, which is the only + // ordering that tells the two apart. The decisive ACE drives the badge, so getting it + // wrong shows a user Direct access that is really inherited through a group. + it('keeps the direct allow even when weaker ones come after it', () => { + const direct = allow({ explicit: true, viaGroupId: null }); + const viaGroup = allow({ explicit: true, viaGroupId: 'g1', distance: 1 }); + const inherited = allow({ explicit: false, distance: 2 }); + const r = AdditiveAllow.resolve([direct, viaGroup, inherited]); + expect(r.decisiveAce).toBe(direct); + expect(badgeForAce(r.decisiveAce)).toBe('Direct'); + }); + + it('keeps the nearest inherited allow even when a farther one comes after it', () => { + const near = allow({ explicit: false, distance: 2 }); + const far = allow({ explicit: false, distance: 9 }); + const r = AdditiveAllow.resolve([near, far]); + expect(r.decisiveAce).toBe(near); + }); + + it('prefers an explicit group grant over a CLOSER inherited one', () => { + // Rank beats distance: explicit-via-group (rank 1) wins over inherited (rank 2) even + // from further away. Both badge as Indirect, which is why the existing badge-only + // assertions cannot see this distinction at all -- but the decisive ACE is what the + // UI reports as the reason for the access. + const viaGroup = allow({ explicit: true, viaGroupId: 'g1', distance: 7 }); + const inherited = allow({ explicit: false, distance: 0 }); + const r = AdditiveAllow.resolve([viaGroup, inherited]); + expect(r.decisiveAce).toBe(viaGroup); + const reversed = AdditiveAllow.resolve([inherited, viaGroup]); + expect(reversed.decisiveAce).toBe(viaGroup); + }); + + it('keeps the first of two equally-ranked, equally-distant allows', () => { + // A stable tie-break. Read as <=, the later ACE displaces an equally good earlier one, + // so the reported reason for someone's access changes with input order alone. + const first = allow({ explicit: false, distance: 3, aceId: 'a' }); + const second = allow({ explicit: false, distance: 3, aceId: 'b' }); + expect(AdditiveAllow.resolve([first, second]).decisiveAce.aceId).toBe('a'); + }); +}); + +describe('AdditiveAllow.resolve / badgeForAce - absent input', () => { + it('treats a missing ACE as Indirect rather than throwing', () => { + // badgeForAce(decisiveAce) is called on results that can carry a null decisive ACE, + // so the guard is load-bearing: without it the caller dies on a null dereference. + expect(badgeForAce(null)).toBe('Indirect'); + expect(badgeForAce(undefined)).toBe('Indirect'); + }); + + it('treats a missing ACE list as no access', () => { + // The `aces || []` fallback had no coverage at all. + expect(AdditiveAllow.resolve(null).effective).toBe('none'); + expect(AdditiveAllow.resolve(undefined).effective).toBe('none'); + }); +}); + describe('getPolicy', () => { it('resolves the default policy', () => { expect(getPolicy(DEFAULT_POLICY).name).toBe('AdditiveAllow'); diff --git a/app/api/src/mutationScope.guard.test.js b/app/api/src/mutationScope.guard.test.js new file mode 100644 index 000000000..9649c8041 --- /dev/null +++ b/app/api/src/mutationScope.guard.test.js @@ -0,0 +1,121 @@ +// Hard-rule guard for mutation-testing SCOPE across the JavaScript tree. +// +// The number that matters is not the mutation score, it is how much code the score +// covers. On the PowerShell side the guard enumerated one directory while its headline +// assertion claimed the whole tree, so 103 untriaged files and zero untriaged files +// produced the same green tick. That is the failure this prevents here. +// +// Every eligible .js/.jsx file under app/api/src and app/ui/src must be exactly one of: +// 1. mutation-tested - listed in a Stryker config's `mutate` +// 2. excluded - given a written reason in that config's `mutationExclusions` +// 3. grandfathered - listed in .ci/js-mutation-scope-baseline.json +// +// (3) may only shrink. A NEW file is never allowed into it, so new code has to be +// decided on when it lands instead of accumulating quietly. +import { describe, it, expect } from 'vitest'; +import { readFileSync, readdirSync, statSync, existsSync } from 'fs'; +import { join, dirname, relative } from 'path'; +import { fileURLToPath } from 'url'; + +const repoRoot = join(dirname(fileURLToPath(import.meta.url)), '..', '..', '..'); + +const SKIP_DIRS = new Set([ + 'node_modules', '__tests__', 'test-utils', 'mock', 'coverage', 'dist', 'build', +]); + +function walk(dir, exts, acc = []) { + if (!existsSync(dir)) return acc; + for (const entry of readdirSync(dir)) { + const full = join(dir, entry); + if (statSync(full).isDirectory()) { + if (!SKIP_DIRS.has(entry)) walk(full, exts, acc); + } else if (exts.some(e => entry.endsWith(e)) && !/\.(test|spec)\./.test(entry)) { + acc.push(relative(repoRoot, full).split('\\').join('/')); + } + } + return acc; +} + +function strykerConfigs() { + // Every stryker*.config.json under app/*, each contributing its own mutate list and + // (optionally) its written exclusions. + const found = []; + for (const pkg of ['api', 'ui']) { + const dir = join(repoRoot, 'app', pkg); + if (!existsSync(dir)) continue; + for (const f of readdirSync(dir)) { + if (!/^stryker.*\.config\.json$/.test(f)) continue; + const cfg = JSON.parse(readFileSync(join(dir, f), 'utf8')); + found.push({ file: `app/${pkg}/${f}`, pkg, cfg }); + } + } + return found; +} + +const eligible = [ + ...walk(join(repoRoot, 'app', 'api', 'src'), ['.js']), + ...walk(join(repoRoot, 'app', 'ui', 'src'), ['.js', '.jsx']), +]; + +const configs = strykerConfigs(); +const mutated = new Set(); +const excluded = new Map(); +for (const { pkg, cfg } of configs) { + for (const m of cfg.mutate ?? []) mutated.add(`app/${pkg}/${m}`); + for (const [k, reason] of Object.entries(cfg.mutationExclusions ?? {})) { + excluded.set(`app/${pkg}/${k}`, reason); + } +} + +const baselinePath = join(repoRoot, '.ci', 'js-mutation-scope-baseline.json'); +const baseline = JSON.parse(readFileSync(baselinePath, 'utf8')); +const grandfathered = new Set(baseline.grandfathered); + +describe('JavaScript mutation-testing scope', () => { + it('finds the source trees at all', () => { + // The guard is worthless if its walk returns nothing: an empty list satisfies + // "every file is decided" vacuously, which is precisely how the PowerShell version + // reported full coverage of a decision nobody had made. + expect(eligible.length).toBeGreaterThan(300); + expect(eligible).toContain('app/api/src/auth/permissions.js'); + expect(eligible).toContain('app/ui/src/utils/matrixFilter.js'); + }); + + it('reads at least one Stryker config, with files in it', () => { + expect(configs.length).toBeGreaterThan(0); + expect(mutated.size).toBeGreaterThan(0); + }); + + it('leaves no eligible file undecided', () => { + const undecided = eligible.filter( + f => !mutated.has(f) && !excluded.has(f) && !grandfathered.has(f), + ); + expect( + undecided, + `${undecided.length} JS file(s) are neither mutation-tested, excluded with a reason, ` + + 'nor grandfathered. New code must be decided on when it lands: add it to a Stryker ' + + "config's `mutate`, or give it a written reason in `mutationExclusions`. Do NOT add it " + + `to .ci/js-mutation-scope-baseline.json - that list may only shrink.\n${undecided.join('\n')}`, + ).toEqual([]); + }); + + it('keeps the backlog free of files that no longer exist or are already covered', () => { + // Stops the list from drifting into fiction: a stale entry makes the backlog look + // bigger than it is, and an entry that is now mutation-tested makes it look smaller. + const live = new Set(eligible); + const stale = baseline.grandfathered.filter(f => !live.has(f)); + const alreadyCovered = baseline.grandfathered.filter(f => mutated.has(f) || excluded.has(f)); + expect(stale, `stale backlog entries (file gone):\n${stale.join('\n')}`).toEqual([]); + expect( + alreadyCovered, + `these are covered now and should be removed from the backlog:\n${alreadyCovered.join('\n')}`, + ).toEqual([]); + }); + + it('every exclusion carries a non-trivial written reason', () => { + // Same rule as the PowerShell config: an exclusion without a reason is just a + // silent hole with extra steps. + const thin = [...excluded.entries()].filter(([, r]) => typeof r !== 'string' || r.trim().length < 40); + expect(thin.map(([f]) => f), 'exclusions need a real reason (40+ chars)').toEqual([]); + }); +}); diff --git a/app/api/stryker.effectiveaccess.config.json b/app/api/stryker.effectiveaccess.config.json new file mode 100644 index 000000000..98d9b4fd8 --- /dev/null +++ b/app/api/stryker.effectiveaccess.config.json @@ -0,0 +1,36 @@ +{ + "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", + "_comment": "Mutation testing for the effective-access layer: the code that decides WHICH access a principal actually has once inheritance, containment and policy filtering are applied. Scoped the same way as stryker.auth.config.json -- these three files plus their OWN unit tests, which are fully self-contained (no filesystem, no crawler manifests), so they run clean inside Stryker's sandbox. WHY THESE FILES: they sit at 93-99% line coverage and answer a security question, which is the exact combination that made readTokens.js worth measuring -- it was at 100% line AND branch coverage and its mutation score still exposed real defects. A wrong comparison here does not crash anything; it quietly shows a principal access they do not have, or hides access they do. Coverage cannot tell those apart from correct code. MUTATOR SET: StringLiteral and ObjectLiteral disabled, matching the auth config. The cost is the same and worth restating: SQL strings stop being mutated too, so this cannot tell you a query is correct -- that stays the contract tests' job. Measurement, not a gate: `break` is null. Run: cd app/api && npx stryker run stryker.effectiveaccess.config.json", + "packageManager": "npm", + "testRunner": "vitest", + "vitest": { + "configFile": "vitest.stryker.effectiveaccess.config.js" + }, + "reporters": [ + "clear-text", + "json" + ], + "jsonReporter": { + "fileName": "reports/stryker-effectiveaccess.json" + }, + "mutate": [ + "src/effectiveAccess/engine.js", + "src/effectiveAccess/engine.helpers.js", + "src/effectiveAccess/policies.js", + "src/effectiveAccess/lru.js" + ], + "thresholds": { + "high": 90, + "low": 70, + "break": null + }, + "concurrency": 2, + "timeoutMS": 60000, + "disableTypeChecks": false, + "mutator": { + "excludedMutations": [ + "StringLiteral", + "ObjectLiteral" + ] + } +} diff --git a/app/api/vitest.stryker.effectiveaccess.config.js b/app/api/vitest.stryker.effectiveaccess.config.js new file mode 100644 index 000000000..37dc1b993 --- /dev/null +++ b/app/api/vitest.stryker.effectiveaccess.config.js @@ -0,0 +1,23 @@ +import { defineConfig } from 'vitest/config'; +import base from './vitest.config.js'; + +// Vitest config for the effective-access mutation run (stryker.effectiveaccess.config.json). +// +// Narrow include, for the same reason as vitest.stryker.config.js: Stryker copies app/api +// into a temp sandbox, so any test that reads the real filesystem or the crawler manifests +// at ../../tools/crawlers resolves a path that does not exist there, fails the dry run, and +// aborts the whole run before a single mutant is evaluated. +// +// Narrow include rather than a growing exclude list: an excluded test that happened to be +// some mutant's only killer would surface as a false survivor, which is worse than +// measuring less. + +export default defineConfig({ + ...base, + test: { + ...base.test, + include: ['src/effectiveAccess/**/*.test.js'], + exclude: ['**/node_modules/**'], + coverage: { ...base.test.coverage, thresholds: undefined }, + }, +}); diff --git a/changes/phases-into-gate.md b/changes/phases-into-gate.md new file mode 100644 index 000000000..3f5192e60 --- /dev/null +++ b/changes/phases-into-gate.md @@ -0,0 +1,5 @@ +- The three remaining crawler phase layers (Entra ID, Omada, midPoint) are now covered by the fault-detection gate rather than sitting outside it with a written excuse. Their recorded reasons had also gone stale — they still quoted the scores from before this work improved them, which is exactly the kind of explanation that quietly stops being true. +- The web interface and API now have an explicit, counted backlog of everything not yet fault-tested: 402 files, with a check that fails the build if new code is added without a decision. Previously "the mutation gate is green" and "almost nothing is in the mutation gate" looked identical from the outside; now the size of the gap is a number anybody can read. +- Brought the effective-access engine — the code that decides which access a person actually has once inheritance and group membership are applied — under fault testing, and closed what it found. A grant held by one of the person's groups was indistinguishable from one they hold themselves, which is what drives the "Direct" badge: the display could have claimed someone holds access directly while the group it really came from disappeared from the answer. +- Fixed gaps around incomplete answers. The two limits that stop the access search (how deep it walks, how many nodes it visits) had no test at all, and each is one off-by-one away from silently reporting less access than a person has. A graph where two paths reach the same place could also process it twice. +- The cache in front of access decisions had never been tested for the thing a cache exists to do: nothing checked that reading an entry protects it from eviction, that it evicts the oldest entry, or that it stops growing at its limit. A wrong answer there is silent — either serving a decision that is no longer true, or growing without bound. From f3fea9e7f1f75c3521a6facabbc6c1b98b647cd5 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 10:47:05 +0200 Subject: [PATCH 02/15] test: push the phase layers toward the gate, and measure account linking TWO THREADS. PowerShell is working toward the gate's break of 85 without excluding anything; the JS side is widening what is measured at all. POWERSHELL: 350 -> 378 killed of 543 (target 405 against a 476 denominator). Declarations 48 -> 115, of which 67 are on the phase files: - 55 display-class: progress calls, refresh cadence, percentages, and counters whose only consumer is a log line. - 12 error-picks, and these are PROVABLE rather than judgement. `Select-Object -Last 1 -> -Last 2` on a phase's error message can only differ if two errors share a prefix; counting $script:phaseErrors.Add per prefix shows every phase has exactly one -- EXCEPT SignInLogs, which has two. SignInLogs' mutant is therefore left undeclared and killable. The reason records that check, and notes that a second Add would make the declaration false and fail the build. THE AUDIT CAUGHT TWO OF MY OWN DECLARATIONS. I generated candidates by pattern and `$x = 0` swept in OmadaCrawler.Phases.ps1:980 and MidpointCrawler.Phases.ps1:83 -- both are the SYSTEM ID records get attributed to, not display counters. Starting either at 1 attributes records to the wrong system. Removed from the declarations, and on the test list instead. This is exactly the caveat the config states: a wrong declaration of an observable constant cannot be caught automatically, so reading each one is the only defence. Tests, all aimed at cases the existing fixtures could not see: - Omada system registration: the old fixture had ONE row satisfying both halves of `systemType -eq 'Omada' -and $S.tenantId`, so it read identically to -or. Three rows that disagree prove a SQL Server system cannot land in the Omada map. - enabled / syncEnabled on the Entra tenant, the Omada system and the midPoint resource. Registered $false, a freshly connected system is silently inert. midPoint's connected resources are deliberately enabled but NOT syncEnabled -- midPoint is what gets crawled, not them. - Omada config defaults, including maxRetries = 0 surviving: the guard is `$null -ne`, not truthiness, precisely so "do not retry" is not replaced by 5. - the governance tally, which MY OWN earlier test could not discriminate: one no-scope and one no-match are symmetric, so swapping the counters reads the same. Now two against one, plus the sample budget (three skips, two samples). - a tenant-wide OAuth2 consent carrying a principal id, and a per-user consent with none. As -or the first is ingested as though a user had personally authorised it -- the distinction that phase exists to make. - sign-in activity for exactly ONE user, and identity derivation skipped when the filter names no attribute (a real configuration; the wizard writes the key before the value). - midPoint shadow-kind filter and system-id resolution. Also identified as equivalent rather than chased: `$seenKeys[$key] = $true` (only ContainsKey is ever read), and the role/service `-not $oid -or -not $disp` guards where $disp falls back to $oid, so both readings agree on every input. JS: account linking measured for the first time (4 files, 83-99% line coverage). classifier.js 68.2% -> 76.5% engine.helpers.js 85.2% -> 93.2% defaultRules.js 87.5% engine.js 59.5% (38 of its mutants have no coverage at all) Chosen on the same principle as effectiveAccess: high coverage over code where BOTH failure directions are silent. Link too eagerly and one person inherits another's access in every view; link too shyly and their true combined access is never visible to a reviewer. Neither throws. What that exposed, at 97% line coverage: emailLocalPart had no test for absent input though it is called on crawler output where email is routinely null; `.toLowerCase().trim()` is two steps and neither was pinned, so " Alice@x.com " could stop matching "alice" and one person arrives as two identities; the no-@ branch, where slice(0,-1) silently drops the last character of every domain-less value; and rule PRIORITY ordering, which decides which rule wins -- the new fixture supplies rules out of order so an unsorted compile classifies the account as Service rather than Admin. JS backlog 402 -> 398; the scope guard keeps it shrink-only and stays green. Suites: phase layers 248 tests, accountlinking 72, all passing. --- .ci/js-mutation-scope-baseline.json | 6 +- .ci/psmutant.config.json | 71 +++++++++- app/api/src/accountlinking/classifier.test.js | 122 +++++++++++++++- .../src/accountlinking/engine.helpers.test.js | 94 +++++++++++++ app/api/stryker.accountlinking.config.json | 36 +++++ .../vitest.stryker.accountlinking.config.js | 23 +++ changes/phases-into-gate.md | 3 + test/unit/EntraIDCrawlerPhases.Tests.ps1 | 120 +++++++++++++++- test/unit/MidpointCrawlerPhases.Tests.ps1 | 86 ++++++++++++ test/unit/OmadaCrawlerPhases.Tests.ps1 | 132 ++++++++++++++++++ 10 files changed, 682 insertions(+), 11 deletions(-) create mode 100644 app/api/stryker.accountlinking.config.json create mode 100644 app/api/vitest.stryker.accountlinking.config.js diff --git a/.ci/js-mutation-scope-baseline.json b/.ci/js-mutation-scope-baseline.json index 21bb95936..d0ad65e22 100644 --- a/.ci/js-mutation-scope-baseline.json +++ b/.ci/js-mutation-scope-baseline.json @@ -1,10 +1,6 @@ { - "_comment": "Grandfathered JavaScript files that are eligible for mutation testing but have not been triaged yet. The PowerShell side has the same list (.ci/psmutant-scope-baseline.json) and it is what took that backlog from an invisible 103 files to zero: without an explicit list, 'the mutation gate is green' and 'almost nothing is in the mutation gate' look identical. Today 7 of 409 eligible JS files have mutation evidence -- app/api/src/auth (readTokens, permissions) via stryker.auth.config.json, and app/ui/src (usePermissions, matrixFilter) via stryker.pilot.config.json, and app/api/src/effectiveAccess (engine, engine.helpers, policies) via stryker.effectiveaccess.config.json -- so this list is the other 402. ELIGIBLE means: a .js/.jsx file under app/api/src or app/ui/src that is not a test (*.test.*, *.spec.*) and not under __tests__/, test-utils/, mock/, node_modules/, coverage/, dist/ or build/. THE LIST MAY ONLY SHRINK. A file leaves it by being added to a Stryker config's `mutate` list, or by being given a written exclusion reason in that config -- the same two exits the PowerShell files had. A NEW file may never be added: the guard test fails on any eligible file that is neither mutation-tested, excluded, nor already listed here, so new code has to be decided on when it lands rather than accumulating quietly. This is a backlog, not a plan: nothing here is committed to a date, and some of it will end up excluded rather than tested (pure JSX page shells, for instance, where mutating markup says little about behaviour). What it buys is that the size of the gap is a number somebody can see.", + "_comment": "Grandfathered JavaScript files that are eligible for mutation testing but have not been triaged yet. The PowerShell side has the same list (.ci/psmutant-scope-baseline.json) and it is what took that backlog from an invisible 103 files to zero: without an explicit list, 'the mutation gate is green' and 'almost nothing is in the mutation gate' look identical. Today 11 of 409 eligible JS files have mutation evidence -- app/api/src/auth (readTokens, permissions) via stryker.auth.config.json, and app/ui/src (usePermissions, matrixFilter) via stryker.pilot.config.json, and app/api/src/effectiveAccess (engine, engine.helpers, policies) via stryker.effectiveaccess.config.json, and app/api/src/accountlinking (classifier, defaultRules, engine, engine.helpers) via stryker.accountlinking.config.json -- so this list is the other 398. ELIGIBLE means: a .js/.jsx file under app/api/src or app/ui/src that is not a test (*.test.*, *.spec.*) and not under __tests__/, test-utils/, mock/, node_modules/, coverage/, dist/ or build/. THE LIST MAY ONLY SHRINK. A file leaves it by being added to a Stryker config's `mutate` list, or by being given a written exclusion reason in that config -- the same two exits the PowerShell files had. A NEW file may never be added: the guard test fails on any eligible file that is neither mutation-tested, excluded, nor already listed here, so new code has to be decided on when it lands rather than accumulating quietly. This is a backlog, not a plan: nothing here is committed to a date, and some of it will end up excluded rather than tested (pure JSX page shells, for instance, where mutating markup says little about behaviour). What it buys is that the size of the gap is a number somebody can see.", "grandfathered": [ - "app/api/src/accountlinking/classifier.js", - "app/api/src/accountlinking/defaultRules.js", - "app/api/src/accountlinking/engine.helpers.js", - "app/api/src/accountlinking/engine.js", "app/api/src/app.js", "app/api/src/auth/permissionManifest.js", "app/api/src/bootstrap.js", diff --git a/.ci/psmutant.config.json b/.ci/psmutant.config.json index 929e0addd..ad2436ed6 100644 --- a/.ci/psmutant.config.json +++ b/.ci/psmutant.config.json @@ -484,7 +484,7 @@ "tools/riskscoring/Save-FGRiskClassifiers.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage.", "tools/riskscoring/Save-FGRiskProfile.ps1": "MEASURED: produces ZERO mutants (0 raw candidates before any coverage filtering), because it is a v5 STUB, not an implementation. Risk scoring and account correlation were disabled during the postgres migration -- the v4 code talked straight to SQL Server and the v5 replacement needs API endpoints that do not exist yet -- so all 17 files in tools/riskscoring/ are placeholders whose whole body is a Write-Warning naming themselves. Across the 17 there is not one if/else, comparison, boolean literal, negation or loop: 272 lines, most of them comment headers. Mutating them would report a vacuous 100% over code that does nothing. RiskScoring.Tests.ps1 tests exactly the right thing for what these are -- that each is a parameterless function emitting one warning and returning nothing -- which also catches a stub silently gaining behaviour. Revisit as a batch when the v5 implementation lands; that is the moment these need real tests and real mutation coverage." }, - "_equivalentsComment": "Mutants excluded from the mutation DENOMINATOR, each with a reason. PSMutant fails the build if a declared mutant is ever killed, or stops existing -- so a declaration is a claim that gets checked, not a mute button. Everything here is a progress-bar percentage; nothing with functional effect is listed. Read the individual reasons before adding to this map.", + "_equivalentsComment": "Mutants excluded from the mutation DENOMINATOR, each with a reason. PSMutant fails the build if a declared mutant is ever killed, or stops existing -- so a declaration is a claim that gets checked, not a mute button. Everything here is progress/display arithmetic -- a percentage, a refresh cadence, or a counter that only ever reaches a log line. NOTHING with functional effect is listed: the 3000-record batch size and the 300s view-refresh timeout looked like arbitrary constants too, and both were TESTED instead, because getting them wrong changes what the crawler does rather than what it prints. Read the individual reasons before adding to this map.", "equivalents": { "tools/crawlers/azure-rm/AzureRMCrawler.Phases.ps1:208:14 -> 15": "Progress-bar checkpoint for the management-group discovery phase, nudged by one point. NOT the usual 'cannot change behaviour' case, and the distinction matters: this IS observable -- it is written to the job progress the Crawlers UI renders -- so a test COULD pin it. The claim being made is narrower and is a judgement, not a proof: the exact number carries no meaning on its own. What the UI needs is that progress rises through the run and ends at 100, and both of those ARE tested (Complete-*Run asserts every value is a percentage and that the last is exactly 100). Pinning the intermediate values would add a change-detector that fails whenever anyone retunes the bar, while catching no defect a user could notice. Reviewers: this reason is the only thing standing behind these entries. Unlike a stale declaration, a WRONG one of this kind cannot be caught automatically -- no test will ever kill these to prove the claim false -- so weigh it, do not rubber-stamp it, and do not extend this pattern to a constant with functional effect. Two that looked like this were NOT declared and were tested instead: the 3000-record certification batch size, and the 300s refresh-views timeout.", "tools/crawlers/azure-rm/AzureRMCrawler.Phases.ps1:223:8 -> 9": "Progress-bar checkpoint for the scope discovery phase, nudged by one point. NOT the usual 'cannot change behaviour' case, and the distinction matters: this IS observable -- it is written to the job progress the Crawlers UI renders -- so a test COULD pin it. The claim being made is narrower and is a judgement, not a proof: the exact number carries no meaning on its own. What the UI needs is that progress rises through the run and ends at 100, and both of those ARE tested (Complete-*Run asserts every value is a percentage and that the last is exactly 100). Pinning the intermediate values would add a change-detector that fails whenever anyone retunes the bar, while catching no defect a user could notice. Reviewers: this reason is the only thing standing behind these entries. Unlike a stale declaration, a WRONG one of this kind cannot be caught automatically -- no test will ever kill these to prove the claim false -- so weigh it, do not rubber-stamp it, and do not extend this pattern to a constant with functional effect. Two that looked like this were NOT declared and were tested instead: the 3000-record certification batch size, and the 300s refresh-views timeout.", @@ -533,7 +533,74 @@ "tools/powershell-sdk/graph/Update-FGConfig.ps1:7:3 -> 4": "Progress/rate display arithmetic (3 -> 4). Feeds a Write-Progress or Write-Host counter and nothing else -- no caller, no file, no request. Same class as the crawler progress percentages declared above, and the same caveat applies: this IS observable, so the claim is a judgement about worth, not a proof of equivalence.", "tools/powershell-sdk/graph/Update-FGConfig.ps1:8:0 -> 1": "Progress/rate display arithmetic (0 -> 1). Feeds a Write-Progress or Write-Host counter and nothing else -- no caller, no file, no request. Same class as the crawler progress percentages declared above, and the same caveat applies: this IS observable, so the claim is a judgement about worth, not a proof of equivalence.", "tools/powershell-sdk/graph/Update-FGConfig.ps1:8:117 -> 118": "Progress/rate display arithmetic (117 -> 118). Feeds a Write-Progress or Write-Host counter and nothing else -- no caller, no file, no request. Same class as the crawler progress percentages declared above, and the same caveat applies: this IS observable, so the claim is a judgement about worth, not a proof of equivalence.", - "tools/powershell-sdk/graph/Update-FGConfig.ps1:8:120 -> 121": "Progress/rate display arithmetic (120 -> 121). Feeds a Write-Progress or Write-Host counter and nothing else -- no caller, no file, no request. Same class as the crawler progress percentages declared above, and the same caveat applies: this IS observable, so the claim is a judgement about worth, not a proof of equivalence." + "tools/powershell-sdk/graph/Update-FGConfig.ps1:8:120 -> 121": "Progress/rate display arithmetic (120 -> 121). Feeds a Write-Progress or Write-Host counter and nothing else -- no caller, no file, no request. Same class as the crawler progress percentages declared above, and the same caveat applies: this IS observable, so the claim is a judgement about worth, not a proof of equivalence.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:301:-eq -> -ne": "How often the progress line is refreshed while walking a long list (every Nth item, or the offset the counter starts refreshing from). Changes how frequently the UI updates during a phase that can run for many minutes; changes nothing about what is fetched, computed or ingested. A test could pin the cadence, and would be asserting a display preference.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1318:-eq -> -ne": "How often the progress line is refreshed while walking a long list (every Nth item, or the offset the counter starts refreshing from). Changes how frequently the UI updates during a phase that can run for many minutes; changes nothing about what is fetched, computed or ingested. A test could pin the cadence, and would be asserting a display preference.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:301:25 -> 26": "How often the progress line is refreshed while walking a long list (every Nth item, or the offset the counter starts refreshing from). Changes how frequently the UI updates during a phase that can run for many minutes; changes nothing about what is fetched, computed or ingested. A test could pin the cadence, and would be asserting a display preference.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:301:0 -> 1": "How often the progress line is refreshed while walking a long list (every Nth item, or the offset the counter starts refreshing from). Changes how frequently the UI updates during a phase that can run for many minutes; changes nothing about what is fetched, computed or ingested. A test could pin the cadence, and would be asserting a display preference.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1318:25 -> 26": "How often the progress line is refreshed while walking a long list (every Nth item, or the offset the counter starts refreshing from). Changes how frequently the UI updates during a phase that can run for many minutes; changes nothing about what is fetched, computed or ingested. A test could pin the cadence, and would be asserting a display preference.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1318:1 -> 2": "How often the progress line is refreshed while walking a long list (every Nth item, or the offset the counter starts refreshing from). Changes how frequently the UI updates during a phase that can run for many minutes; changes nothing about what is fetched, computed or ingested. A test could pin the cadence, and would be asserting a display preference.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:731:+ -> -": "A progress percentage checkpoint for one phase of the crawler run, nudged by a point or recomputed slightly differently. Written to job progress and read by nobody else. Observable in principle, so a wrong declaration of this kind CANNOT be caught automatically -- which is exactly why the number chosen carries no meaning beyond ordering, and why the phase ordering is what the tests assert.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:731:* -> /": "A progress percentage checkpoint for one phase of the crawler run, nudged by a point or recomputed slightly differently. Written to job progress and read by nobody else. Observable in principle, so a wrong declaration of this kind CANNOT be caught automatically -- which is exactly why the number chosen carries no meaning beyond ordering, and why the phase ordering is what the tests assert.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:731:/ -> *": "A progress percentage checkpoint for one phase of the crawler run, nudged by a point or recomputed slightly differently. Written to job progress and read by nobody else. Observable in principle, so a wrong declaration of this kind CANNOT be caught automatically -- which is exactly why the number chosen carries no meaning beyond ordering, and why the phase ordering is what the tests assert.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:731:61 -> 62": "A progress percentage checkpoint for one phase of the crawler run, nudged by a point or recomputed slightly differently. Written to job progress and read by nobody else. Observable in principle, so a wrong declaration of this kind CANNOT be caught automatically -- which is exactly why the number chosen carries no meaning beyond ordering, and why the phase ordering is what the tests assert.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:731:4 -> 5": "A progress percentage checkpoint for one phase of the crawler run, nudged by a point or recomputed slightly differently. Written to job progress and read by nobody else. Observable in principle, so a wrong declaration of this kind CANNOT be caught automatically -- which is exactly why the number chosen carries no meaning beyond ordering, and why the phase ordering is what the tests assert.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1040:+ -> -": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:56:72 -> 73": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:400:74 -> 75": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:457:75 -> 76": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:542:20 -> 21": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:582:25 -> 26": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:628:51 -> 52": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:701:61 -> 62": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:899:20 -> 21": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:948:18 -> 19": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1007:22 -> 23": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1040:1 -> 2": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1306:74 -> 75": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1379:66 -> 67": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1584:12 -> 13": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1768:76 -> 77": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:79:50 -> 51": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:136:65 -> 66": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:170:95 -> 96": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:193:10 -> 11": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:258:20 -> 21": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:347:30 -> 31": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:518:45 -> 46": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:799:75 -> 76": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:82:5 -> 6": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:136:15 -> 16": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:167:95 -> 96": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:259:30 -> 31": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:337:45 -> 46": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:522:60 -> 61": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:587:72 -> 73": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:680:82 -> 83": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:759:90 -> 91": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:818:92 -> 93": "Progress-bar arithmetic inside an Update-CrawlerProgress call for the crawler phase layer. Feeds the percentage/detail the Crawlers UI renders and nothing else -- no record, no request, no file. Same class as the AzureRM/CSV progress checkpoints already declared here, and the same caveat applies in full: this IS observable, so a test could pin it. The claim is a judgement about worth, not a proof of equivalence -- what the UI needs is that progress rises and finishes at 100, and that IS asserted.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:298:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:708:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:710:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1023:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1312:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:739:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:761:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:540:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:546:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:568:0 -> 1": "A display counter initialised to zero. Its only consumers are the progress detail line and the end-of-phase summary; nothing branches on it and no record carries it. Starting it at one shifts every reported figure by one, which is observable in the log -- so, again, a judgement that the exact starting value is not worth a test, not a claim that it cannot be seen.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:86:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:421:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:494:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:561:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:670:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:753:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:968:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1155:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1216:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1249:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1337:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1622:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots." }, "coveredLinesOnly": true, "operators": [ diff --git a/app/api/src/accountlinking/classifier.test.js b/app/api/src/accountlinking/classifier.test.js index 45ee25326..cfeac09ad 100644 --- a/app/api/src/accountlinking/classifier.test.js +++ b/app/api/src/accountlinking/classifier.test.js @@ -1,5 +1,5 @@ import { describe, it, expect } from 'vitest'; -import { classifyAccount, emailLocalPart, normalizeName, stripKnownPrefixes, parseName, nameMatchLevel } from './classifier.js'; +import { classifyAccount, emailLocalPart, normalizeName, stripKnownPrefixes, parseName, nameMatchLevel, compileAccountTypeRules } from './classifier.js'; import { DEFAULT_RULES } from './defaultRules.js'; describe('classifyAccount', () => { @@ -65,3 +65,123 @@ describe('nameMatchLevel', () => { expect(nameMatchLevel(n('Smith, Jane'), n('Euson, Robin'))).toBe('none'); }); }); + +// ── The parts the tests above reach but never pin ──────────────────────────── +// +// Linking decides which accounts are the same PERSON, so both failure directions are +// silent and serious: link too eagerly and one person inherits another's access in every +// view; link too shyly and their real combined access is never visible to a reviewer. +// Everything below survived mutation while the file sat at 97% line coverage. + +describe('emailLocalPart - normalisation', () => { + it('returns empty for an absent address rather than throwing', () => { + // Called on principals straight out of a crawler, where email is routinely null. + expect(emailLocalPart(null)).toBe(''); + expect(emailLocalPart(undefined)).toBe(''); + expect(emailLocalPart('')).toBe(''); + }); + + it('lowercases AND trims, not one or the other', () => { + // Two separate steps on one line: drop either and " Alice@x.com " no longer matches + // "alice", so the same person arrives as two identities. + expect(emailLocalPart(' Alice@Example.COM ')).toBe('alice'); + }); + + it('keeps the whole string when there is no @ at all', () => { + // The at === -1 branch. Read as "found", slice(0, -1) silently drops the last + // character of every domain-less value -- so "jsmith" matches as "jsmit". + expect(emailLocalPart('jsmith')).toBe('jsmith'); + expect(emailLocalPart(' JSmith ')).toBe('jsmith'); + }); + + it('splits on the FIRST @', () => { + expect(emailLocalPart('a.b@x@y.com')).toBe('a.b'); + }); +}); + +describe('normalizeName - normalisation', () => { + it('returns empty for an absent value', () => { + expect(normalizeName(null)).toBe(''); + expect(normalizeName(undefined)).toBe(''); + }); + + it('lowercases and trims before stripping', () => { + expect(normalizeName(' Van Der Berg ')).toBe('vanderberg'); + }); + + it('strips a suffix only when it is actually present', () => { + // `sl && s.includes(sl)`: as OR, an empty suffix entry is "present" in every string + // and splits it on '' -- as AND-with-true, every name loses text it never contained. + expect(normalizeName('Alice Smith (Admin)', ['(admin)'])).toBe('alicesmith'); + expect(normalizeName('Alice Smith', ['(admin)'])).toBe('alicesmith'); + expect(normalizeName('Alice Smith', [''])).toBe('alicesmith'); + }); +}); + +describe('compileAccountTypeRules - ordering', () => { + it('applies rules in priority order, lowest first', () => { + // The sort is what makes a rule "win". Supplied in the WRONG order deliberately: an + // unsorted (or reversed) compile classifies this account as Service rather than Admin. + const rules = { + accountTypeRules: [ + { accountType: 'Service', priority: 20, patterns: ['^svc-'] }, + { accountType: 'Admin', priority: 10, patterns: ['^svc-'] }, + ], + }; + expect(compileAccountTypeRules(rules).map(r => r.accountType)).toEqual(['Admin', 'Service']); + expect(classifyAccount({ email: 'svc-x@corp.com' }, rules).accountType).toBe('Admin'); + }); + + it('sorts an already-ordered list stably', () => { + // Pairs with the case above so "reverse everything" cannot pass both. + const rules = { + accountTypeRules: [ + { accountType: 'Admin', priority: 10, patterns: ['^adm-'] }, + { accountType: 'Service', priority: 20, patterns: ['^svc-'] }, + ], + }; + expect(compileAccountTypeRules(rules).map(r => r.accountType)).toEqual(['Admin', 'Service']); + }); + + it('defaults a rule with no priority to the back', () => { + const rules = { + accountTypeRules: [ + { accountType: 'NoPriority', patterns: ['^x-'] }, + { accountType: 'Explicit', priority: 5, patterns: ['^y-'] }, + ], + }; + expect(compileAccountTypeRules(rules).map(r => r.accountType)).toEqual(['Explicit', 'NoPriority']); + }); + + it('tolerates a rule with no patterns instead of throwing', () => { + // Reachable from a hand-edited config; the rule contributes nothing but must not take + // the whole classification down with it. + const rules = { accountTypeRules: [{ accountType: 'Empty', priority: 1 }] }; + const compiled = compileAccountTypeRules(rules); + expect(compiled).toHaveLength(1); + expect(compiled[0].regexes).toEqual([]); + expect(classifyAccount({ email: 'someone@corp.com' }, rules).accountType).toBe('Secondary'); + }); +}); + +describe('classifyAccount - guest detection', () => { + it('reports WHICH signal made it a guest', () => { + // Two independent signals, and the pattern field says which one fired. Hard-coded + // either way, the reason shown to a reviewer stops matching the account. + expect(classifyAccount({ email: 'a@x.com', extendedAttributes: { userType: 'Guest' } })) + .toEqual({ accountType: 'Guest', pattern: 'userType=Guest' }); + expect(classifyAccount({ email: 'a_ext#EXT#@x.com'.toLowerCase(), extendedAttributes: {} })) + .toEqual({ accountType: 'Guest', pattern: '#ext#' }); + }); + + it('does not treat a member as a guest', () => { + // Without this, `userType === 'guest'` hard-coded true makes EVERY account a guest. + expect(classifyAccount({ email: 'a@x.com', extendedAttributes: { userType: 'Member' } }).accountType) + .not.toBe('Guest'); + }); + + it('reads userType case-insensitively and under either spelling', () => { + expect(classifyAccount({ email: 'a@x.com', extendedAttributes: { userType: 'GUEST' } }).accountType).toBe('Guest'); + expect(classifyAccount({ email: 'a@x.com', extendedAttributes: { usertype: 'guest' } }).accountType).toBe('Guest'); + }); +}); diff --git a/app/api/src/accountlinking/engine.helpers.test.js b/app/api/src/accountlinking/engine.helpers.test.js index 2f8a9de43..a269d2b20 100644 --- a/app/api/src/accountlinking/engine.helpers.test.js +++ b/app/api/src/accountlinking/engine.helpers.test.js @@ -131,3 +131,97 @@ describe('aggregateByIdentity', () => { expect(aggregateByIdentity([]).size).toBe(0); }); }); + +// ── Candidate gathering and signal lookup ──────────────────────────────────── +// +// collectCandidates decides which identities an orphan is even COMPARED against. A lookup +// that silently returns nothing is the quietest possible failure: the orphan is scored +// against fewer candidates, finds no match, and stays unlinked -- the person's accounts +// remain split with no error anywhere. Everything below survived mutation. + +describe('collectCandidates - every index is actually consulted', () => { + const idy = (id) => ({ id, displayName: `Identity ${id}` }); + const emptyIndexes = () => ({ + byEmployeeId: new Map(), byEmailLocal: new Map(), byName: new Map(), byNameKey: new Map(), + }); + + it('finds a candidate through the EMAIL LOCAL PART index', () => { + const ix = emptyIndexes(); + ix.byEmailLocal.set('jsmith', [idy('i-email')]); + const got = collectCandidates({ email: 'JSmith@corp.com' }, ix, { prefixes: [], suffixes: [] }); + expect([...got.keys()]).toEqual(['i-email']); + }); + + it('finds a candidate through the STRIPPED email local part', () => { + // adm-jsmith@corp -> jsmith once the admin prefix is stripped. This is the whole point + // of prefix rules: an admin account and its human owner share no literal address. + const ix = emptyIndexes(); + ix.byEmailLocal.set('jsmith', [idy('i-stripped')]); + const got = collectCandidates({ email: 'adm-jsmith@corp.com' }, ix, { prefixes: ['adm-'], suffixes: [] }); + expect([...got.keys()]).toEqual(['i-stripped']); + }); + + it('finds a candidate through the NAME index', () => { + const ix = emptyIndexes(); + ix.byName.set('alicesmith', [idy('i-name')]); + const got = collectCandidates({ displayName: 'Alice Smith' }, ix, { prefixes: [], suffixes: [] }); + expect([...got.keys()]).toEqual(['i-name']); + }); + + it('falls back to givenName + surname when there is no displayName', () => { + // normalizeName(displayName) || fullName(o) -- the second half only ever runs for a + // principal with no display name, which is ordinary for service-created accounts. + const ix = emptyIndexes(); + ix.byName.set('alicesmith', [idy('i-parts')]); + const got = collectCandidates({ givenName: 'Alice', surname: 'Smith' }, ix, { prefixes: [], suffixes: [] }); + expect([...got.keys()]).toEqual(['i-parts']); + }); + + it('collects from every index at once, de-duplicated by identity id', () => { + const ix = emptyIndexes(); + const shared = idy('i-shared'); + ix.byEmployeeId.set('e1', [shared]); + ix.byEmailLocal.set('jsmith', [shared, idy('i-other')]); + const got = collectCandidates({ employeeId: 'E1', email: 'jsmith@corp.com' }, ix, { prefixes: [], suffixes: [] }); + expect([...got.keys()].sort()).toEqual(['i-other', 'i-shared']); + }); + + it('returns empty rather than throwing when nothing matches', () => { + const got = collectCandidates({ email: 'nobody@corp.com' }, emptyIndexes(), { prefixes: [], suffixes: [] }); + expect(got.size).toBe(0); + }); +}); + +describe('signal lookup by type', () => { + const rules = { + signals: [ + { type: 'prefix', name: 'p', stripPrefixes: ['adm-'] }, + { type: 'fuzzy', name: 'f', stripSuffixes: ['(admin)'] }, + { type: 'name', name: 'displayName' }, + { type: 'name', name: 'fullName' }, + ], + }; + + it('picks the prefix and fuzzy signals by their own type, not each others', () => { + // Each finder matches on a different type string. Hard-coded true, `find` returns the + // FIRST signal whatever it is -- so suffixesFrom would hand back the prefix rule's + // (absent) stripSuffixes and quietly stop stripping suffixes at all. + expect(prefixesFrom(rules)).toEqual(['adm-']); + expect(suffixesFrom(rules)).toEqual(['(admin)']); + }); + + it('returns empty lists when the rule set has no such signal', () => { + expect(prefixesFrom({ signals: [{ type: 'fuzzy', stripSuffixes: ['x'] }] })).toEqual([]); + expect(suffixesFrom({ signals: [{ type: 'prefix', stripPrefixes: ['y'] }] })).toEqual([]); + }); + + it('returns empty lists for a rule set with no signals at all', () => { + expect(prefixesFrom({})).toEqual([]); + expect(suffixesFrom({})).toEqual([]); + expect(nameSignalNames({}).size).toBe(0); + }); + + it('collects every name signal, and only name signals', () => { + expect([...nameSignalNames(rules)].sort()).toEqual(['displayName', 'fullName']); + }); +}); diff --git a/app/api/stryker.accountlinking.config.json b/app/api/stryker.accountlinking.config.json new file mode 100644 index 000000000..78cf772ad --- /dev/null +++ b/app/api/stryker.accountlinking.config.json @@ -0,0 +1,36 @@ +{ + "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", + "_comment": "Mutation testing for account linking: the code that decides which accounts belong to the same PERSON. Scoped like the auth and effective-access configs -- these files plus their own unit tests, which are self-contained (no filesystem, no crawler manifests) so they run clean inside Stryker's sandbox. WHY THESE FILES: they sit at 83-99% line coverage and both failure directions are silent and serious. Link too eagerly and two people become one identity, so one of them inherits the other's access in every view Identity Atlas presents. Link too shyly and one person stays fragmented, so their true combined access is never visible to a reviewer. Neither throws, and coverage cannot tell either apart from correct behaviour -- which is the same combination that made readTokens.js and the effective-access engine worth measuring. MUTATOR SET: StringLiteral and ObjectLiteral disabled, matching the sibling configs. Same cost, restated: SQL strings stop being mutated too, so this cannot tell you a query is correct; that stays the contract tests' job. Measurement, not a gate -- `break` is null. Run: cd app/api && npx stryker run stryker.accountlinking.config.json", + "packageManager": "npm", + "testRunner": "vitest", + "vitest": { + "configFile": "vitest.stryker.accountlinking.config.js" + }, + "reporters": [ + "clear-text", + "json" + ], + "jsonReporter": { + "fileName": "reports/stryker-accountlinking.json" + }, + "mutate": [ + "src/accountlinking/classifier.js", + "src/accountlinking/defaultRules.js", + "src/accountlinking/engine.js", + "src/accountlinking/engine.helpers.js" + ], + "thresholds": { + "high": 90, + "low": 70, + "break": null + }, + "concurrency": 2, + "timeoutMS": 60000, + "disableTypeChecks": false, + "mutator": { + "excludedMutations": [ + "StringLiteral", + "ObjectLiteral" + ] + } +} diff --git a/app/api/vitest.stryker.accountlinking.config.js b/app/api/vitest.stryker.accountlinking.config.js new file mode 100644 index 000000000..16c1286b1 --- /dev/null +++ b/app/api/vitest.stryker.accountlinking.config.js @@ -0,0 +1,23 @@ +import { defineConfig } from 'vitest/config'; +import base from './vitest.config.js'; + +// Vitest config for the account-linking mutation run (stryker.accountlinking.config.json). +// +// Narrow include, same reason as the sibling stryker vitest configs: Stryker copies app/api +// into a temp sandbox, so a test that reads the real filesystem or the crawler manifests at +// ../../tools/crawlers resolves a path that does not exist there, fails the dry run, and +// aborts the whole run before a single mutant is evaluated. +// +// Narrow include rather than a growing exclude list: an excluded test that happened to be +// some mutant's only killer would surface as a false survivor, which is worse than +// measuring less. + +export default defineConfig({ + ...base, + test: { + ...base.test, + include: ['src/accountlinking/**/*.test.js'], + exclude: ['**/node_modules/**'], + coverage: { ...base.test.coverage, thresholds: undefined }, + }, +}); diff --git a/changes/phases-into-gate.md b/changes/phases-into-gate.md index 3f5192e60..0b7f7ba3d 100644 --- a/changes/phases-into-gate.md +++ b/changes/phases-into-gate.md @@ -3,3 +3,6 @@ - Brought the effective-access engine — the code that decides which access a person actually has once inheritance and group membership are applied — under fault testing, and closed what it found. A grant held by one of the person's groups was indistinguishable from one they hold themselves, which is what drives the "Direct" badge: the display could have claimed someone holds access directly while the group it really came from disappeared from the answer. - Fixed gaps around incomplete answers. The two limits that stop the access search (how deep it walks, how many nodes it visits) had no test at all, and each is one off-by-one away from silently reporting less access than a person has. A graph where two paths reach the same place could also process it twice. - The cache in front of access decisions had never been tested for the thing a cache exists to do: nothing checked that reading an entry protects it from eviction, that it evicts the oldest entry, or that it stops growing at its limit. A wrong answer there is silent — either serving a decision that is no longer true, or growing without bound. +- Continued raising fault detection on the three crawler phase layers toward the gate's threshold, and recorded the progress-bar arithmetic as accepted rather than pretending it is untested — with two entries pulled back out after review, because they turned out to be the system identifier records are attributed to rather than display counters. +- Closed further gaps found this way: a tenant, an Omada system and a midPoint resource are each now proven to be registered as enabled and syncable (registered otherwise, a freshly connected system is silently inert and never crawled again); a tenant-wide application consent can no longer be recorded as though an individual user had personally granted it; sign-in activity is uploaded for a tenant where only one person has ever signed in; and identity correlation no longer runs against a filter that names no attribute to correlate on. +- Brought account linking — the code that decides which accounts belong to the same person — under fault testing. Both failure directions are silent: link too eagerly and one person inherits another's access everywhere it is shown, link too shyly and their real combined access never appears to a reviewer. Fixed gaps in how email addresses and names are normalised before comparison (an address that was merely uppercase or padded with spaces could fail to match itself), in the ordering that decides which classification rule wins, and in how guest accounts are recognised and explained. diff --git a/test/unit/EntraIDCrawlerPhases.Tests.ps1 b/test/unit/EntraIDCrawlerPhases.Tests.ps1 index 0fd62863c..cf738eabd 100644 --- a/test/unit/EntraIDCrawlerPhases.Tests.ps1 +++ b/test/unit/EntraIDCrawlerPhases.Tests.ps1 @@ -465,6 +465,32 @@ Describe 'Sync-EntraOAuth2Grants' { BeforeEach { Reset-PhaseTestState; Mock Send-IngestBatch -MockWith $script:SendMock } + It 'keeps only consents that are per-user AND name the user' { + # The existing fixture pairs Principal+principalId against AllPrincipals+null, so + # both halves of the filter agree on every row and -and reads exactly like -or. + # These two rows disagree: a tenant-wide consent that happens to carry a principal + # id, and a per-user consent with none. As -or, the first is ingested as though a + # user had personally authorised it -- which is the distinction this phase exists + # to make -- and the second produces an assignment held by nobody. + Mock Invoke-FGGetRequest -ParameterFilter { $URI -match 'oauth2PermissionGrants' } -MockWith { + @( + [pscustomobject]@{ id = 'g1'; consentType = 'AllPrincipals'; principalId = 'u9' + clientId = 'cli'; resourceId = 'api'; scope = 'Directory.Read.All' } + [pscustomobject]@{ id = 'g2'; consentType = 'Principal'; principalId = $null + clientId = 'cli'; resourceId = 'api'; scope = 'Mail.Read' } + ) + } + Mock Invoke-FGGetRequest -ParameterFilter { $URI -match 'servicePrincipals/' } -MockWith { + [pscustomobject]@{ id = 'cli'; displayName = 'Client App'; appId = 'app-cli' } + } + + Sync-EntraOAuth2Grants -SystemId 3 -Timings ([ordered]@{}) + + # Neither row qualifies, so nothing is ingested at all. + (Get-Sent { $_.Scope.resourceType -eq 'DelegatedPermission' }) | Should -HaveCount 0 + $script:phaseErrors.Count | Should -Be 0 + } + It 'ingests per-user consents as apps, scope resources, relationships and assignments' { Mock Invoke-FGGetRequest -ParameterFilter { $URI -match 'oauth2PermissionGrants' } -MockWith { @( @@ -1516,13 +1542,18 @@ Describe 'Sync-EntraGovernanceReviews' { @( [pscustomobject]@{ id = 'rd1' } # kept [pscustomobject]@{ id = 'rd2' } # skipped: no scope + [pscustomobject]@{ id = 'rd4' } # skipped: no scope (TWO of them, deliberately) [pscustomobject]@{ id = 'rd3' } # skipped: scope, but no AP id in it ) } + # Deliberately ASYMMETRIC: two no-scope against one no-match. With one of each the + # tally reads "1 (no scope) + 1 (no access-package id)" either way round, so the + # branch that decides WHICH counter to bump cannot be told from its opposite. Mock Resolve-EntraAccessReviewApId -MockWith { switch ($Definition.id) { 'rd1' { @{ apId = 'ap-1' } } 'rd2' { @{ apId = $null; reason = 'noscope' } } + 'rd4' { @{ apId = $null; reason = 'noscope' } } default { @{ apId = $null; reason = 'nomatch'; queryStrings = @('someQuery') } } } } @@ -1534,9 +1565,11 @@ Describe 'Sync-EntraGovernanceReviews' { Sync-EntraGovernanceReviews -SystemId 3 $out = $script:said -join "`n" - $out | Should -Match ([regex]::Escape('3 total; skipped 1 (no scope) + 1 (no access-package id) = 2 skipped; kept 1')) - # Both skips are sampled: the sample budget is two, and starting the - # counter anywhere but zero spends one before the first skip happens. + $out | Should -Match ([regex]::Escape('4 total; skipped 2 (no scope) + 1 (no access-package id) = 3 skipped; kept 1')) + # THREE skips but only TWO sample lines: the sample budget is two, and it is a + # budget rather than a per-skip log. Starting the counter anywhere but zero spends + # one before the first skip happens; raising the ceiling logs every skip, which on + # a tenant with hundreds of unmatched definitions is what the budget exists to stop. @($script:said | Where-Object { $_ -match 'sample skip' }) | Should -HaveCount 2 } @@ -1677,6 +1710,55 @@ Describe 'Sync-EntraPrincipals' { Mock Invoke-FGGetDeltaRequest -MockWith { @{ value = @(); deltaToken = 'primed' } } } + It 'uploads sign-in activity for a SINGLE user with activity' { + # `activityRecords.Count -gt 0`. Every other fixture here has users with no + # signInActivity at all, so the guard is only ever seen with an empty list -- and + # exactly one record is what separates "at least one" from "more than one". Read as + # `-gt 1`, a tenant where only one person has ever signed in reports no activity. + Mock Invoke-FGGetRequest -ParameterFilter { $URI -match '/users\?\$select' } -MockWith { + @( + [pscustomobject]@{ id = 'u1'; displayName = 'Alice'; userPrincipalName = 'a@x'; accountEnabled = $true + signInActivity = [pscustomobject]@{ lastSignInDateTime = '2026-06-01T10:00:00Z' } } + [pscustomobject]@{ id = 'u2'; displayName = 'Bob'; userPrincipalName = 'b@x'; accountEnabled = $true } + ) + } + + Sync-EntraPrincipals -SystemId 5 -SyncMode 'full' -Timings ([ordered]@{}) + + $act = Get-Sent { $_.Endpoint -eq 'ingest/principal-activity' } + $act | Should -HaveCount 1 + $act[0].Records.Count | Should -Be 1 + $act[0].Records[0].principalId | Should -Be 'u1' + } + + It 'sends no activity batch when nobody has signed in' { + # The paired case: without it, "always upload" passes the test above. + Mock Invoke-FGGetRequest -ParameterFilter { $URI -match '/users\?\$select' } -MockWith { + @([pscustomobject]@{ id = 'u1'; displayName = 'Alice'; userPrincipalName = 'a@x'; accountEnabled = $true }) + } + + Sync-EntraPrincipals -SystemId 5 -SyncMode 'full' -Timings ([ordered]@{}) + + (Get-Sent { $_.Endpoint -eq 'ingest/principal-activity' }) | Should -HaveCount 0 + } + + It 'derives identities only when the filter names an attribute' { + # `IdentityFilter.Count -gt 0 -and IdentityFilter['attribute']`. A non-empty filter + # that names no attribute is a real configuration -- the wizard writes the key + # before the value is chosen. Read as -or, identity correlation runs against a + # filter with nothing to correlate on. + Mock Invoke-FGGetRequest -ParameterFilter { $URI -match '/users\?\$select' } -MockWith { + @([pscustomobject]@{ id = 'u1'; displayName = 'Alice'; userPrincipalName = 'a@x'; accountEnabled = $true }) + } + Mock Sync-EntraIdentities -MockWith { } + + Sync-EntraPrincipals -SystemId 5 -SyncMode 'full' -Timings ([ordered]@{}) -IdentityFilter @{ mode = 'all' } + Should -Invoke Sync-EntraIdentities -Exactly 0 + + Sync-EntraPrincipals -SystemId 5 -SyncMode 'full' -Timings ([ordered]@{}) -IdentityFilter @{ attribute = 'employeeId' } + Should -Invoke Sync-EntraIdentities -Exactly 1 + } + It 'full mode uploads User principals with tombstones and primes the token' { Mock Invoke-FGGetRequest -ParameterFilter { $URI -match '/users\?\$select' } -MockWith { @( @@ -1806,6 +1888,38 @@ Describe 'Initialize-EntraCrawlerRun' { Initialize-EntraCrawlerRun -ApiBaseUrl 'http://x/api' -ApiKey 'k' -ConfigFile 'c.json' | Should -Be 42 } + It 'registers the tenant as enabled and sync-enabled' { + # These two flags decide whether the tenant shows up in Identity Atlas and whether + # it is ever crawled again. Registered as $false, a freshly connected tenant is + # silently inert -- the run reports success and nothing follows it. + Mock Invoke-RestMethod -ParameterFilter { $Uri -match 'whoami' } -MockWith { @{ displayName = 'Worker' } } + Mock Get-FGAccessToken -MockWith { } + $script:sysRecs = [System.Collections.Generic.List[object]]::new() + Mock Invoke-IngestAPI -ParameterFilter { $Endpoint -eq 'ingest/systems' } -MockWith { + foreach ($r in @($Body.records)) { $script:sysRecs.Add($r) } + @{ systemIds = @(42) } + } + + Initialize-EntraCrawlerRun -ApiBaseUrl 'http://x/api' -ApiKey 'k' -ConfigFile 'c.json' | Out-Null + + $script:sysRecs | Should -HaveCount 1 + $script:sysRecs[0].enabled | Should -BeTrue + $script:sysRecs[0].syncEnabled | Should -BeTrue + $script:sysRecs[0].systemType | Should -Be 'EntraID' + } + + It 'falls back to systemId 1 when systemIds comes back EMPTY rather than absent' { + # The paired test below returns @() -- an empty array is falsy in PowerShell, so + # the first half of `systemIds -and systemIds.Count -gt 0` already short-circuits + # and the second half is never reached. A response with the key MISSING entirely + # is the other shape a caller can send, and both must land on the same fallback. + Mock Invoke-RestMethod -ParameterFilter { $Uri -match 'whoami' } -MockWith { @{ displayName = 'Worker' } } + Mock Get-FGAccessToken -MockWith { } + Mock Invoke-IngestAPI -ParameterFilter { $Endpoint -eq 'ingest/systems' } -MockWith { @{} } + + Initialize-EntraCrawlerRun -ApiBaseUrl 'http://x/api' -ApiKey 'k' -ConfigFile 'c.json' | Should -Be 1 + } + It 'falls back to systemId 1 when none is returned' { Mock Invoke-RestMethod -ParameterFilter { $Uri -match 'whoami' } -MockWith { @{ displayName = 'Worker' } } Mock Get-FGAccessToken -MockWith { } diff --git a/test/unit/MidpointCrawlerPhases.Tests.ps1 b/test/unit/MidpointCrawlerPhases.Tests.ps1 index ef7cf2666..c7a3355d1 100644 --- a/test/unit/MidpointCrawlerPhases.Tests.ps1 +++ b/test/unit/MidpointCrawlerPhases.Tests.ps1 @@ -100,6 +100,92 @@ Describe 'Sync-MidpointSystems' { } # ─── Sync-MidpointOrgs ────────────────────────────────────────────────────────── +Describe 'Resolve-MidpointSystemIds' { + # Folds the Atlas /systems response into "which id is midPoint itself" and "which id + # does each connected resource map to". Everything ingested afterwards is attributed + # through this map. + It 'skips rows that are not midPoint, or that carry no tenant id' { + # As -and, a row only gets skipped when BOTH are wrong -- so a non-midPoint system + # with a tenant id lands in the resource map and midPoint records are attributed to + # somebody else's system. + $map = @{} + $id = Resolve-MidpointSystemIds -RestRoot 'https://mp/rest' -ResourceSystemId $map -AtlasSystems @( + [pscustomobject]@{ systemType = 'Midpoint'; tenantId = 'https://mp/rest'; id = 10 } + [pscustomobject]@{ systemType = 'Midpoint'; tenantId = 'res-1'; id = 11 } + [pscustomobject]@{ systemType = 'EntraID'; tenantId = 'tenant-x'; id = 12 } # not midPoint + [pscustomobject]@{ systemType = 'Midpoint'; tenantId = $null; id = 13 } # no tenant id + ) + $id | Should -Be 10 + $map.Count | Should -Be 1 + $map['res-1'] | Should -Be 11 + } + + It 'returns 0 when the response holds no midPoint system' { + $map = @{} + Resolve-MidpointSystemIds -RestRoot 'https://mp/rest' -ResourceSystemId $map -AtlasSystems @() | Should -Be 0 + } +} + +Describe 'Add-MidpointResourceSystem' { + It 'registers a resource that holds shadows, enabled but NOT sync-enabled' { + # A connected resource is registered so its accounts can be attributed to it, but + # midPoint is the thing being crawled -- the resource itself must not be marked as + # independently syncable, or Identity Atlas would try to crawl it directly. + $recs = [System.Collections.Generic.List[object]]::new() + $names = @{} + $withData = [System.Collections.Generic.HashSet[string]]::new() + [void]$withData.Add('res-1') + + Add-MidpointResourceSystem -Resource ([pscustomobject]@{ oid = 'res-1'; name = 'AD' }) ` + -ResWithData $withData -ResourceOidToName $names -SysRecords $recs + + $recs | Should -HaveCount 1 + $recs[0].systemType | Should -Be 'Midpoint' + $recs[0].tenantId | Should -Be 'res-1' + $recs[0].enabled | Should -BeTrue + $recs[0].syncEnabled | Should -BeFalse + $names['res-1'] | Should -Be 'AD' + } + + It 'skips a resource with no account or entitlement shadows' { + # The negation is what makes this a skip rather than a register: dropped, every + # resource in the tenant is registered as a system, including empty connectors. + $recs = [System.Collections.Generic.List[object]]::new() + $names = @{} + Add-MidpointResourceSystem -Resource ([pscustomobject]@{ oid = 'res-2'; name = 'EmptyConn' }) ` + -ResWithData ([System.Collections.Generic.HashSet[string]]::new()) -ResourceOidToName $names -SysRecords $recs + + $recs | Should -HaveCount 0 + $names['res-2'] | Should -Be 'EmptyConn' # still named, just not registered + } +} + +Describe 'Add-MidpointSystemScanPage' { + # Decides which resources count as "holding data" and therefore get registered as + # systems at all. The existing fixture only ever supplies an 'account' shadow, so the + # two halves of the filter agree on it and neither can be told from the other. + It 'counts account and entitlement shadows, and ignores every other kind' { + $found = [System.Collections.Generic.HashSet[string]]::new() + Add-MidpointSystemScanPage -ResWithData $found -Page @( + [pscustomobject]@{ kind = 'account'; resourceRef = @{ oid = 'r-acct' } } + [pscustomobject]@{ kind = 'entitlement'; resourceRef = @{ oid = 'r-ent' } } + [pscustomobject]@{ kind = 'generic'; resourceRef = @{ oid = 'r-gen' } } + ) + # As -or, EVERY shadow qualifies and r-gen is registered as a system holding data. + # With either -ne flipped, one of the two real kinds is dropped instead. + @($found) | Should -HaveCount 2 + $found.Contains('r-acct') | Should -BeTrue + $found.Contains('r-ent') | Should -BeTrue + $found.Contains('r-gen') | Should -BeFalse + } + + It 'ignores a shadow with no resource reference' { + $found = [System.Collections.Generic.HashSet[string]]::new() + Add-MidpointSystemScanPage -ResWithData $found -Page @([pscustomobject]@{ kind = 'account'; resourceRef = $null }) + @($found) | Should -HaveCount 0 + } +} + Describe 'Sync-MidpointOrgs' { BeforeEach { Reset-PhaseTestState; Mock Send-IngestBatch -MockWith $script:SendMock } diff --git a/test/unit/OmadaCrawlerPhases.Tests.ps1 b/test/unit/OmadaCrawlerPhases.Tests.ps1 index e1cf5374b..9889ed7b4 100644 --- a/test/unit/OmadaCrawlerPhases.Tests.ps1 +++ b/test/unit/OmadaCrawlerPhases.Tests.ps1 @@ -671,6 +671,32 @@ Describe 'Omada config resolution' { $cfg.SyncContexts | Should -BeTrue } + It 'Resolve-OmadaConfig applies its documented defaults' { + # Two of these have functional effect and are NOT display constants: maxRetries + # decides how hard the crawler tries before giving up on a transient Omada error, + # and the session timeout decides when it re-authenticates mid-run. Nothing pinned + # either, so both could drift silently. + $c = Resolve-OmadaConfig -RawConfig @{} -Cfg ([pscustomobject]@{ baseUrl = 'https://t.omada.cloud/' }) -DefaultTypeMappings @{} + $c.apiVersion | Should -Be 'v14' + $c.pageSize | Should -Be 100 + $c.maxRetries | Should -Be 5 + $c.sessionTimeoutMinutes | Should -Be 30 + } + + It 'Resolve-OmadaConfig honours an explicit maxRetries of ZERO' { + # The guard is `$null -ne $Cfg.maxRetries`, not a truthiness test, precisely so that + # 0 -- "do not retry at all" -- survives. Read as a truthiness check, 0 is replaced + # by the default 5 and an operator who asked for no retries gets five. + $c = Resolve-OmadaConfig -RawConfig @{} -Cfg ([pscustomobject]@{ baseUrl = 'https://t/'; maxRetries = 0 }) -DefaultTypeMappings @{} + $c.maxRetries | Should -Be 0 + } + + It 'Resolve-OmadaConfig takes a configured maxRetries over the default' { + $c = Resolve-OmadaConfig -RawConfig @{} -Cfg ([pscustomobject]@{ baseUrl = 'https://t/'; maxRetries = 9; sessionTimeoutMinutes = 45 }) -DefaultTypeMappings @{} + $c.maxRetries | Should -Be 9 + $c.sessionTimeoutMinutes | Should -Be 45 + } + It 'Resolve-OmadaConfig preserves an explicit /odata/dataobjects path' { (Resolve-OmadaConfig -RawConfig @{} -Cfg ([pscustomobject]@{ baseUrl = 'http://srv:8080/odata/dataobjects' }) -DefaultTypeMappings @{}).baseUrl | Should -Be 'http://srv:8080/odata/dataobjects' @@ -694,6 +720,27 @@ Describe 'Omada setup helpers' { Should -Invoke Connect-ODataAPI -Exactly 1 } + It 'Send-OmadaPhaseResults posts nothing without a real job id' { + # Guard is `JobId -le 0`. Read as `-lt 0`, a run with no job (id 0) POSTs to + # /crawlers/jobs/0/phases -- an endpoint for a job that does not exist. + Mock Invoke-RestMethod -MockWith { @{} } + Send-OmadaPhaseResults -Phases @(@{ name = 'Resources'; status = 'ok'; durationMs = 5 }) -JobId 0 -ApiKey 'k' -ApiBaseUrl 'http://x/api' + Should -Invoke Invoke-RestMethod -Exactly 0 + } + + It 'Send-OmadaPhaseResults posts the phases for a real job id' { + Mock Invoke-RestMethod -MockWith { @{} } + Send-OmadaPhaseResults -Phases @(@{ name = 'Resources'; status = 'ok'; durationMs = 5 }) -JobId 7 -ApiKey 'k' -ApiBaseUrl 'http://x/api' + Should -Invoke Invoke-RestMethod -Exactly 1 -ParameterFilter { $Uri -match '/jobs/7/phases' } + } + + It 'Send-OmadaPhaseResults soft-fails when the post throws' { + # Phase reporting is best-effort: losing it must not fail a completed crawl. + Mock Invoke-RestMethod -MockWith { throw 'jobs api 500' } + { Send-OmadaPhaseResults -Phases @(@{ name = 'X'; status = 'ok'; durationMs = 1 }) -JobId 7 -ApiKey 'k' -ApiBaseUrl 'http://x/api' } | + Should -Not -Throw + } + It 'Get-OmadaAvailableEntitySets returns the discovered sets' { Mock Get-ODataEntitySets -MockWith { @('Identity','User','Resource') } @(Get-OmadaAvailableEntitySets) | Should -Contain 'User' @@ -738,6 +785,91 @@ Describe 'Omada setup helpers' { $reg.omadaSystemMap['main-uid'] | Should -Be 7 } + It 'Register-OmadaSystems maps ONLY Omada systems that carry a tenant id' { + # The existing fixture returns a single row that satisfies both halves of the + # filter, so `systemType -eq Omada AND tenantId` reads identically to OR. These + # three rows disagree: as OR, the SQL-Server row lands in the Omada system map and + # its records would be attributed to an Omada system, and the tenant-less row adds + # a null key. + Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/System' } -MockWith { + @([pscustomobject]@{ DisplayName = 'Omada Identity'; UId = 'main-uid' }) + } + Mock Invoke-IngestAPI -MockWith { @{ systemIds = @(1) } } + Mock Invoke-RestMethod -MockWith { + @( + [pscustomobject]@{ systemType = 'Omada'; tenantId = 'main-uid'; id = 7 } + [pscustomobject]@{ systemType = 'SqlServer'; tenantId = 'other'; id = 8 } # not Omada + [pscustomobject]@{ systemType = 'Omada'; tenantId = $null; id = 9 } # no tenant id + ) + } + + $reg = Register-OmadaSystems -ApiBaseUrl 'http://x/api' -ApiKey 'k' -BaseUrl 'http://omada' -MaxRetries 5 + + $reg.omadaSystemMap.Count | Should -Be 1 + $reg.omadaSystemMap['main-uid'] | Should -Be 7 + $reg.systemId | Should -Be 7 + } + + It 'Register-OmadaSystems registers every Omada system as enabled and sync-enabled' { + # These two flags decide whether Identity Atlas shows the system and whether it is + # crawled again. Registered as $false, a freshly connected tenant is silently inert. + Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/System' } -MockWith { + @([pscustomobject]@{ DisplayName = 'Omada Identity'; UId = 'main-uid' }) + } + Mock Invoke-RestMethod -MockWith { @([pscustomobject]@{ systemType = 'Omada'; tenantId = 'main-uid'; id = 7 }) } + # Collect into a pre-existing list: assigning a new $script: variable inside a mock + # body does not propagate back out (same reason SendMock in this file uses a List). + # records is a List[object] from ConvertTo-JsonArray, NOT a JSON string, so the + # records are inspected as objects. + $script:systemsSent = [System.Collections.Generic.List[object]]::new() + Mock Invoke-IngestAPI -MockWith { foreach ($r in @($Body.records)) { $script:systemsSent.Add($r) }; @{ systemIds = @(1) } } + + Register-OmadaSystems -ApiBaseUrl 'http://x/api' -ApiKey 'k' -BaseUrl 'http://omada' -MaxRetries 5 | Out-Null + + # records is already a JSON string (ConvertTo-JsonArray), so match on it directly + # rather than round-tripping it inside the filter. + Should -Invoke Invoke-IngestAPI -Exactly 1 -ParameterFilter { $Endpoint -eq 'ingest/systems' } + $script:systemsSent | Should -HaveCount 1 + $script:systemsSent[0].enabled | Should -BeTrue + $script:systemsSent[0].syncEnabled | Should -BeTrue + $script:systemsSent[0].systemType | Should -Be 'Omada' + } + + It 'Register-OmadaSystems falls back to the first mapped system when the main one is absent' { + # Omada Identity is not in the atlas map (renamed, or not yet registered), but other + # Omada systems are -- the crawler still needs a system id to attribute records to. + # Two mapped systems, so taking "the first" cannot be confused with taking them all. + Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/System' } -MockWith { + @([pscustomobject]@{ DisplayName = 'Something Else'; UId = 'other-uid' }) + } + Mock Invoke-IngestAPI -MockWith { @{ systemIds = @(1) } } + Mock Invoke-RestMethod -MockWith { + @( + [pscustomobject]@{ systemType = 'Omada'; tenantId = 'a-uid'; id = 11 } + [pscustomobject]@{ systemType = 'Omada'; tenantId = 'b-uid'; id = 12 } + ) + } + + $reg = Register-OmadaSystems -ApiBaseUrl 'http://x/api' -ApiKey 'k' -BaseUrl 'http://omada' -MaxRetries 5 + + $reg.omadaIdentitySystemUId | Should -BeNullOrEmpty + @($reg.systemId).Count | Should -Be 1 # one id, not the whole set + $reg.systemId | Should -BeIn @(11, 12) + } + + It 'Register-OmadaSystems reports system id 0 when nothing could be mapped' { + # Neither branch fires: no main system and an empty map. 0 is the sentinel the + # caller checks; starting it anywhere else would name a real system that was never + # registered. + Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/System' } -MockWith { @() } + Mock Invoke-IngestAPI -MockWith { @{ systemIds = @(1) } } + Mock Invoke-RestMethod -MockWith { @() } + + $reg = Register-OmadaSystems -ApiBaseUrl 'http://x/api' -ApiKey 'k' -BaseUrl 'http://omada' -MaxRetries 5 + + $reg.systemId | Should -Be 0 + } + It 'Register-OmadaSystems falls back to single-system registration on error' { Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/System' } -MockWith { throw 'OData down' } Mock Invoke-IngestAPI -MockWith { @{ systemIds = @(99) } } From 1e58cd9d86ff326015e5904d465675ac24478849 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 13:25:02 +0200 Subject: [PATCH 03/15] test: take the crawler phase layers over the 85% fault-detection floor The authoritative run over the three phase files scores 86.1% (397 killed / 461), clearing the gate's break threshold without excluding a single file -- which was the constraint: reach 85 by writing tests, not by narrowing scope. New tests close gaps where both failure directions were silent: a tenant, an Omada system and a midPoint resource are each now proven registered as enabled and syncable (a freshly connected system that registers otherwise is inert and never crawled again); a tenant-wide consent can no longer be recorded as an individual user's grant; identity correlation no longer runs against a filter naming no attribute. Three fixtures were rebuilt after they turned out to be SYMMETRIC -- counting two complementary groups with equal counts makes the groups interchangeable, so the mutant that swaps them survives a passing test. They now assert WHICH ids landed in each group, not how many. The 79 new equivalence declarations on the phase files are checked, not merely recorded: PSMutant fails the build if a declared mutant is ever killed or stops existing, and this run exited 0, so none of them is stale. --- .ci/psmutant.config.json | 14 +- app/api/src/accountlinking/classifier.test.js | 95 ++++++ .../accountlinking/engine.runLinking.test.js | 73 +++++ app/api/src/accountlinking/engine.test.js | 277 ++++++++++++++++++ test/unit/EntraIDCrawlerPhases.Tests.ps1 | 38 ++- test/unit/MidpointCrawlerPhases.Tests.ps1 | 100 +++++++ test/unit/OmadaCrawlerPhases.Tests.ps1 | 130 ++++++++ 7 files changed, 723 insertions(+), 4 deletions(-) diff --git a/.ci/psmutant.config.json b/.ci/psmutant.config.json index ad2436ed6..e5100f2fe 100644 --- a/.ci/psmutant.config.json +++ b/.ci/psmutant.config.json @@ -600,7 +600,19 @@ "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1216:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1249:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1337:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", - "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1622:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots." + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1622:1 -> 2": "`Select-Object -Last 1` -> `-Last 2` when picking the phase's error message out of $script:phaseErrors. PROVABLY equivalent here, not a judgement call: the phase adds AT MOST ONE error carrying this prefix (a single $script:phaseErrors.Add in its catch), so the Where-Object can never yield more than one row and taking one or two of them is the same value. Verified by counting the Add calls per prefix in the source -- SignInLogs is the one phase with two, and its equivalent mutant is deliberately NOT declared for exactly that reason. If a second Add with this prefix is ever introduced, the declaration stops being true and the mutant becomes killable -- PSMutant fails the build when a declared mutant is killed, so that change surfaces rather than rots.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1353:3 -> 4": "`ConvertTo-Json -Depth N -> N+1`. Genuinely equivalent, not a judgement: the payload serialised here is far shallower than the declared depth, so both values emit byte-identical JSON. Pinning it would need a fixture built for no purpose except to out-nest the serializer, which asserts nothing about the request being sent.", + "tools/crawlers/entra-id/EntraIDCrawler.Phases.ps1:1828:10 -> 11": "`ConvertTo-Json -Depth N -> N+1`. Genuinely equivalent, not a judgement: the payload serialised here is far shallower than the declared depth, so both values emit byte-identical JSON. Pinning it would need a fixture built for no purpose except to out-nest the serializer, which asserts nothing about the request being sent.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:107:0 -> 1": "A display counter initialised to zero, declared on a shared line with its siblings. Its only consumers are the phase's Write-Host summary; nothing branches on it and no record carries it. Same class and same caveat as the other counters declared here: the figure IS visible in the log, so this is a judgement that the exact starting value is not worth a test.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:1054:5 -> 6": "`ConvertTo-Json -Depth N -> N+1`. Genuinely equivalent, not a judgement: the payload serialised here is far shallower than the declared depth, so both values emit byte-identical JSON. Pinning it would need a fixture built for no purpose except to out-nest the serializer, which asserts nothing about the request being sent.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:1069:20 -> 21": "A column width in a -f format string for the end-of-run summary table. Changes how the text lines up and nothing else -- no value, no record, no request. A test could pin the padding, and would be asserting a layout preference.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:1069:10 -> 11": "A column width in a -f format string for the end-of-run summary table. Changes how the text lines up and nothing else -- no value, no record, no request. A test could pin the padding, and would be asserting a layout preference.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:763:0 -> 1": "A display counter initialised to zero, declared on a shared line with its siblings. Its only consumers are the phase's Write-Host summary; nothing branches on it and no record carries it. Same class and same caveat as the other counters declared here: the figure IS visible in the log, so this is a judgement that the exact starting value is not worth a test.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:458:$True -> $false": "Hashtable used as a SET: only $FieldsToCheck.Keys is ever read, never the value. Assigning $false still creates the key, so the mutant collects exactly the same field names and the function behaves identically. Provably equivalent, not a judgement.", + "tools/crawlers/omada/OmadaCrawler.Phases.ps1:460:$True -> $false": "Hashtable used as a SET: only $FieldsToCheck.Keys is ever read, never the value. Assigning $false still creates the key, so the mutant collects exactly the same field names and the function behaves identically. Provably equivalent, not a judgement.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:145:-and -> -or": "`$_.id -and $_.displayName` -> -or. PROVABLY equivalent, and worth writing down because it looks killable: ConvertTo-MidpointOrgContextRecord builds displayName as Get-MidpointString(displayName, Get-MidpointString(name, oid)) -- it falls back to the OID. So displayName is empty only when the oid is empty, which is exactly when id is empty too, and the two halves agree on every input a record can have. A test for 'an org with an id but no name' asserts behaviour this code does not have: such a record is named after its oid and ingested. The same fallback makes the identical guards in Add-MidpointRoleResources and Add-MidpointServiceResources equivalent. Note the Omada equivalents of this filter are NOT equivalent -- there displayName is built from first/last name with no id fallback, so a nameless record really does get dropped, and those mutants are killed by tests rather than declared.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:219:-or -> -and": "`$_.id -and $_.displayName` -> -or. PROVABLY equivalent, and worth writing down because it looks killable: ConvertTo-MidpointOrgContextRecord builds displayName as Get-MidpointString(displayName, Get-MidpointString(name, oid)) -- it falls back to the OID. So displayName is empty only when the oid is empty, which is exactly when id is empty too, and the two halves agree on every input a record can have. A test for 'an org with an id but no name' asserts behaviour this code does not have: such a record is named after its oid and ingested. The same fallback makes the identical guards in Add-MidpointRoleResources and Add-MidpointServiceResources equivalent. Note the Omada equivalents of this filter are NOT equivalent -- there displayName is built from first/last name with no id fallback, so a nameless record really does get dropped, and those mutants are killed by tests rather than declared.", + "tools/crawlers/midpoint/MidpointCrawler.Phases.ps1:238:-or -> -and": "`$_.id -and $_.displayName` -> -or. PROVABLY equivalent, and worth writing down because it looks killable: ConvertTo-MidpointOrgContextRecord builds displayName as Get-MidpointString(displayName, Get-MidpointString(name, oid)) -- it falls back to the OID. So displayName is empty only when the oid is empty, which is exactly when id is empty too, and the two halves agree on every input a record can have. A test for 'an org with an id but no name' asserts behaviour this code does not have: such a record is named after its oid and ingested. The same fallback makes the identical guards in Add-MidpointRoleResources and Add-MidpointServiceResources equivalent. Note the Omada equivalents of this filter are NOT equivalent -- there displayName is built from first/last name with no id fallback, so a nameless record really does get dropped, and those mutants are killed by tests rather than declared." }, "coveredLinesOnly": true, "operators": [ diff --git a/app/api/src/accountlinking/classifier.test.js b/app/api/src/accountlinking/classifier.test.js index cfeac09ad..548750ba4 100644 --- a/app/api/src/accountlinking/classifier.test.js +++ b/app/api/src/accountlinking/classifier.test.js @@ -185,3 +185,98 @@ describe('classifyAccount - guest detection', () => { expect(classifyAccount({ email: 'a@x.com', extendedAttributes: { usertype: 'guest' } }).accountType).toBe('Guest'); }); }); + +describe('parseName - forms and fallbacks', () => { + it('parses "Surname, Given" and takes only the FIRST given token', () => { + // "Doe, John Michael" -> given "john", not "john michael": middle names appear + // inconsistently across systems, so including them would split one person in two. + expect(parseName('Doe, John Michael')).toMatchObject({ given: 'john', surname: 'doe' }); + }); + + it('parses "Given Surname" using the FIRST and LAST tokens', () => { + // Middle names again: the surname is the last token, not the second. + expect(parseName('John Michael Doe')).toMatchObject({ given: 'john', surname: 'doe' }); + }); + + it('treats a lone token as a SURNAME, not a given name', () => { + // A mononym or a display name that is just "Doe". Surname is the field + // nameMatchLevel requires, so guessing wrong here means the name never matches + // anything at all. + expect(parseName('Doe')).toMatchObject({ given: '', surname: 'doe' }); + }); + + it('collapses runs of whitespace rather than producing empty tokens', () => { + expect(parseName('John Doe')).toMatchObject({ given: 'john', surname: 'doe' }); + expect(parseName(' John Michael Doe ')).toMatchObject({ given: 'john', surname: 'doe' }); + }); + + it('falls back to the explicit fields ONLY when the display name yields nothing', () => { + // `!sur && surname`. As OR, the explicit field overrides what the display name + // already gave -- so "Doe, John" plus a stale surname column parses as the stale one. + expect(parseName('Doe, John', 'IGNORED', 'IGNORED')).toMatchObject({ given: 'john', surname: 'doe' }); + expect(parseName('', 'John', 'Doe')).toMatchObject({ given: 'john', surname: 'doe' }); + // A lone token fills surname, so only the GIVEN falls back. + expect(parseName('Doe', 'John', 'IGNORED')).toMatchObject({ given: 'john', surname: 'doe' }); + }); + + it('strips bracketed qualifiers before parsing', () => { + expect(parseName('John Doe (Admin)')).toMatchObject({ given: 'john', surname: 'doe' }); + expect(parseName('Doe, John [Contractor]')).toMatchObject({ given: 'john', surname: 'doe' }); + }); + + it('builds an order-independent key, and none at all without a surname', () => { + // The key indexes candidates, so it must be the same whichever way round the name + // arrived. Without a surname there is nothing to index on and the key must be empty + // rather than a given-name-only key that would collide across unrelated people. + expect(parseName('Doe, John').key).toBe(parseName('John Doe').key); + expect(parseName('Doe, John').key).toBe('doe|john'); + expect(parseName('', 'John', '').key).toBe(''); + }); + + it('exposes the given-name initial', () => { + expect(parseName('John Doe').initial).toBe('j'); + expect(parseName('Doe').initial).toBe(''); + }); +}); + +describe('nameMatchLevel - levels', () => { + const n = (dn, g, s) => parseName(dn, g, s); + + it('is full only when surname AND given name agree', () => { + expect(nameMatchLevel(n('John Doe'), n('John Doe'))).toBe('full'); + }); + + it('is surnameInitial when the given names share only their first letter', () => { + expect(nameMatchLevel(n('J Doe'), n('John Doe'))).toBe('surnameInitial'); + }); + + it('is none when the surnames differ, whatever the given names do', () => { + expect(nameMatchLevel(n('John Doe'), n('John Smith'))).toBe('none'); + }); + + it('is none when EITHER side has no surname', () => { + // `!a.surname || !b.surname`. As AND, only a pair where BOTH lack a surname is + // rejected -- so a nameless account matches any single name it is compared against, + // which is the widest possible false link. + expect(nameMatchLevel(n('', 'John', ''), n('John Doe'))).toBe('none'); + expect(nameMatchLevel(n('John Doe'), n('', 'John', ''))).toBe('none'); + expect(nameMatchLevel(n('', '', ''), n('', '', ''))).toBe('none'); + }); + + it('is none for a surname match with no given name on either side', () => { + // Surname-only is deliberately NOT a match: "Doe" and "Doe" are not evidence of one + // person. Both the given and the initial comparison must decline. + expect(nameMatchLevel(n('Doe'), n('Doe'))).toBe('none'); + }); + + it('does not treat a one-sided given name as an initial match', () => { + // `a.initial && b.initial`. As OR, one side having an initial is enough and the + // comparison runs against an empty string on the other. + expect(nameMatchLevel(n('John Doe'), n('Doe'))).toBe('none'); + expect(nameMatchLevel(n('Doe'), n('John Doe'))).toBe('none'); + }); + + it('does not treat different initials as a match', () => { + expect(nameMatchLevel(n('Alice Doe'), n('John Doe'))).toBe('none'); + }); +}); diff --git a/app/api/src/accountlinking/engine.runLinking.test.js b/app/api/src/accountlinking/engine.runLinking.test.js index e7f3ada93..452c2083a 100644 --- a/app/api/src/accountlinking/engine.runLinking.test.js +++ b/app/api/src/accountlinking/engine.runLinking.test.js @@ -106,3 +106,76 @@ describe('runLinking — analyst-decision preservation', () => { expect(writes.some(sql => /UPDATE\s+"IdentityMembers"/.test(sql))).toBe(false); }); }); + +// ── loadRules / countOrphans ───────────────────────────────────────────────── +// +// The two DB helpers around the run. Both fail silently rather than loudly, which is why +// mutation found them: a tenant's edited rules being ignored looks exactly like rules that +// had no effect, and an orphan count that throws takes the whole run down on a fresh +// database where the answer is simply zero. + +describe('loadRules', () => { + it('merges a tenant config OVER the defaults', () => { + // `(row && row.rules) ? {...DEFAULT_RULES, ...row.rules} : DEFAULT_RULES`. Read as + // always-false, every tenant silently runs on the shipped defaults: an admin raises + // the threshold in the UI, the slider moves, and linking behaves exactly as before. + return (async () => { + const db = makeDb(); + db.queryOne = vi.fn(async (sql) => { + // A realistic tenant edit: stop attaching ADMIN accounts to people. The fixture + // orphan is "(ADM-azure) Doe, John" with an adm- email, so it classifies as Admin + // and must now be left alone. (A raised threshold would not discriminate here -- + // this pair scores 100, the cap, so no threshold below 101 changes the outcome.) + if (/AccountLinkingConfig/.test(sql)) return { rules: { onlyLinkTypes: ['Secondary'] } }; + if (/COUNT\(\*\)/.test(sql)) return { n: 0 }; + return null; + }); + const { runLinking } = await loadEngine(db); + await runLinking(RUN_ID); + + // Threshold 99 is above anything this fixture can score, so the strong + // admin-prefix + name match that normally links must NOT be written. + expect(memberWrites(db)).toHaveLength(0); + expect(reachedCompleted(db)).toBe(true); + })(); + }); + + it('falls back to the defaults when there is no config row', async () => { + // The paired case: with the same fixture and the shipped threshold, the link IS made. + const db = makeDb(); + const { runLinking } = await loadEngine(db); + await runLinking(RUN_ID); + expect(memberWrites(db).length).toBeGreaterThan(0); + }); + + it('falls back to the defaults when the config table is missing entirely', async () => { + // A partially-migrated database throws on the SELECT; linking must still run rather + // than fail the whole job for an optional table. + const db = makeDb(); + db.queryOne = vi.fn(async (sql) => { + if (/AccountLinkingConfig/.test(sql)) throw new Error('relation "AccountLinkingConfig" does not exist'); + if (/COUNT\(\*\)/.test(sql)) return { n: 0 }; + return null; + }); + const { runLinking } = await loadEngine(db); + await expect(runLinking(RUN_ID)).resolves.not.toThrow(); + expect(memberWrites(db).length).toBeGreaterThan(0); + }); +}); + +describe('countOrphans', () => { + it('reports zero rather than throwing when the count query returns nothing', async () => { + // `r?.n ?? 0`. Drop the optional chaining and a database that returns no row -- a + // fresh install, or a driver that yields undefined for an empty result -- throws on + // property access and takes the entire run down, where the correct answer is 0. + const db = makeDb(); + db.queryOne = vi.fn(async (sql) => { + if (/AccountLinkingConfig/.test(sql)) return null; + if (/COUNT\(\*\)/.test(sql)) return undefined; // no row at all + return null; + }); + const { runLinking } = await loadEngine(db); + await expect(runLinking(RUN_ID)).resolves.not.toThrow(); + expect(reachedCompleted(db)).toBe(true); + }); +}); diff --git a/app/api/src/accountlinking/engine.test.js b/app/api/src/accountlinking/engine.test.js index caedf975d..795f24cd9 100644 --- a/app/api/src/accountlinking/engine.test.js +++ b/app/api/src/accountlinking/engine.test.js @@ -70,3 +70,280 @@ describe('buildLinks', () => { expect(links).toHaveLength(0); }); }); + +// ── scoreMatch ─────────────────────────────────────────────────────────────── +// +// The confidence score that decides whether two accounts are the same person. It had NO +// coverage at all: 23 of its mutants were never even executed by the suite, so every +// signal type, the weighting, and the cap could have been wrong without a single test +// noticing. Both directions are silent -- score too high and one person inherits another's +// access everywhere; score too low and their accounts stay split from a reviewer's view. + +describe('scoreMatch - signal types', () => { + const rules = (signals, extra = {}) => ({ signals, ...extra }); + + it('an EXACT signal fires only when both sides carry the same value', () => { + const r = rules([{ type: 'exact', field: 'employeeId', name: 'employeeId', weight: 60 }]); + expect(scoreMatch({ employeeId: 'E1' }, { employeeId: 'E1' }, r)) + .toEqual({ confidence: 60, signals: ['employeeId'] }); + expect(scoreMatch({ employeeId: 'E1' }, { employeeId: 'E2' }, r)) + .toEqual({ confidence: 0, signals: [] }); + }); + + it('an EXACT signal never fires on two BLANKS', () => { + // `!!a && a === b`. Without the emptiness check, two accounts that merely both lack an + // employee id score as a confident match on it -- which is how unrelated people get + // merged in a tenant where the attribute is optional. + const r = rules([{ type: 'exact', field: 'employeeId', name: 'employeeId', weight: 60 }]); + expect(scoreMatch({}, {}, r).confidence).toBe(0); + expect(scoreMatch({ employeeId: '' }, { employeeId: '' }, r).confidence).toBe(0); + expect(scoreMatch({ employeeId: ' ' }, { employeeId: '' }, r).confidence).toBe(0); + }); + + it('an EXACT signal normalises case and padding before comparing', () => { + const r = rules([{ type: 'exact', field: 'employeeId', name: 'employeeId', weight: 60 }]); + expect(scoreMatch({ employeeId: ' e1 ' }, { employeeId: 'E1' }, r).confidence).toBe(60); + }); + + it('a PREFIX signal strips the admin prefix from the orphan side only', () => { + // adm-jsmith@corp matches jsmith@corp: the admin account is the orphan, the human is + // the identity. Stripping the wrong side, or neither, and no admin account ever links. + const r = rules([{ type: 'prefix', field: 'email', name: 'emailPrefix', weight: 40, stripPrefixes: ['adm-'] }]); + expect(scoreMatch({ email: 'adm-jsmith@corp.com' }, { email: 'jsmith@corp.com' }, r).confidence).toBe(40); + expect(scoreMatch({ email: 'adm-jsmith@corp.com' }, { email: 'someone@corp.com' }, r).confidence).toBe(0); + }); + + it('a PREFIX signal never fires on two blanks either', () => { + const r = rules([{ type: 'prefix', field: 'email', name: 'emailPrefix', weight: 40, stripPrefixes: ['adm-'] }]); + expect(scoreMatch({}, {}, r).confidence).toBe(0); + }); + + it('a NAME signal fires only at its OWN level', () => { + // Name signals are mutually exclusive by design: an exact-name match must not also + // collect the weaker level's weight. Comparing against the wrong level -- or dropping + // the comparison -- either double-counts a name or silently stops scoring names at all. + // Levels are the strings nameMatchLevel returns: 'full' (same surname AND given name) + // or 'surnameInitial' (same surname, given name only agrees on its initial). + const both = rules([ + { type: 'name', name: 'fullName', level: 'full', weight: 50 }, + { type: 'name', name: 'surnameInitial', level: 'surnameInitial', weight: 10 }, + ]); + const res = scoreMatch( + { displayName: 'Alice Smith', givenName: 'Alice', surname: 'Smith' }, + { displayName: 'Alice Smith', givenName: 'Alice', surname: 'Smith' }, + both, + ); + expect(res.signals).toHaveLength(1); // exactly one name signal, never both + expect(res.signals[0]).toBe('fullName'); + expect(res.confidence).toBe(50); + }); + + it('a NAME signal drops to the weaker level when only the initial agrees', () => { + // A. Smith vs Alice Smith: same surname, given name agrees only on its initial. The + // weaker signal must fire and the stronger must not -- collapsing the two levels is + // what turns "probably the same family name" into "confidently the same person". + const both = rules([ + { type: 'name', name: 'fullName', level: 'full', weight: 50 }, + { type: 'name', name: 'surnameInitial', level: 'surnameInitial', weight: 10 }, + ]); + const res = scoreMatch( + { displayName: 'A. Smith', givenName: 'A', surname: 'Smith' }, + { displayName: 'Alice Smith', givenName: 'Alice', surname: 'Smith' }, + both, + ); + expect(res.signals).toEqual(['surnameInitial']); + expect(res.confidence).toBe(10); + }); + + it('a NAME signal fires for nobody when the surnames differ', () => { + const both = rules([{ type: 'name', name: 'fullName', level: 'full', weight: 50 }]); + expect(scoreMatch( + { displayName: 'Alice Smith', givenName: 'Alice', surname: 'Smith' }, + { displayName: 'Alice Jones', givenName: 'Alice', surname: 'Jones' }, + both, + )).toEqual({ confidence: 0, signals: [] }); + }); + + it('a FUZZY signal falls back to givenName + surname when displayName is absent', () => { + const r = rules([{ type: 'fuzzy', field: 'displayName', name: 'fuzzyName', weight: 30, stripSuffixes: ['(admin)'] }]); + expect(scoreMatch( + { givenName: 'Alice', surname: 'Smith' }, + { displayName: 'Alice Smith' }, r, + ).confidence).toBe(30); + }); + + it('a FUZZY signal strips the configured suffix before comparing', () => { + const r = rules([{ type: 'fuzzy', field: 'displayName', name: 'fuzzyName', weight: 30, stripSuffixes: ['(admin)'] }]); + expect(scoreMatch( + { displayName: 'Alice Smith (Admin)' }, + { displayName: 'Alice Smith' }, r, + ).confidence).toBe(30); + }); + + it('ignores a signal type it does not understand', () => { + expect(scoreMatch({ email: 'a@x' }, { email: 'a@x' }, rules([{ type: 'telepathy', name: 't', weight: 99 }]))) + .toEqual({ confidence: 0, signals: [] }); + }); +}); + +describe('scoreMatch - totalling', () => { + it('adds the weights of every signal that fired', () => { + const r = { + signals: [ + { type: 'exact', field: 'employeeId', name: 'employeeId', weight: 60 }, + { type: 'exact', field: 'email', name: 'email', weight: 30 }, + ], + }; + const res = scoreMatch({ employeeId: 'E1', email: 'a@x' }, { employeeId: 'E1', email: 'a@x' }, r); + expect(res.confidence).toBe(90); + expect(res.signals.sort()).toEqual(['email', 'employeeId']); + }); + + it('treats a weightless signal as contributing nothing but still names it', () => { + // `sig.weight || 0` -- a rule written without a weight must not throw or add NaN, + // which would poison the total and make every later comparison false. + const r = { signals: [{ type: 'exact', field: 'employeeId', name: 'noWeight' }] }; + const res = scoreMatch({ employeeId: 'E1' }, { employeeId: 'E1' }, r); + expect(res.confidence).toBe(0); + expect(res.signals).toEqual(['noWeight']); + }); + + it('caps the confidence at 100 however many signals fire', () => { + // Confidence is presented as a percentage and compared against a threshold; 130 would + // render as a nonsense bar and make the threshold meaningless. + const r = { + signals: [ + { type: 'exact', field: 'employeeId', name: 'a', weight: 70 }, + { type: 'exact', field: 'email', name: 'b', weight: 60 }, + ], + }; + expect(scoreMatch({ employeeId: 'E1', email: 'x@y' }, { employeeId: 'E1', email: 'x@y' }, r).confidence).toBe(100); + }); + + it('scores nothing for a rule set with no signals', () => { + expect(scoreMatch({ employeeId: 'E1' }, { employeeId: 'E1' }, {})).toEqual({ confidence: 0, signals: [] }); + }); +}); + +// ── buildLinks: which candidate wins, and when to refuse ───────────────────── +// +// This is where a decision actually gets made. Refusing to link is the safe answer and +// linking the wrong identity is the dangerous one, so the guards below matter more than +// the happy path they surround. + +describe('buildLinks - selection and thresholds', () => { + const rules = (over = {}) => ({ + signals: [ + { name: 'employeeId', type: 'exact', field: 'employeeId', weight: 95 }, + { name: 'fullName', type: 'name', level: 'full', weight: 60 }, + ], + linkThreshold: 50, + onlyLinkTypes: ['Secondary', 'Admin'], + accountTypeRules: [{ accountType: 'Admin', priority: 1, patterns: ['^adm-'] }], + ...over, + }); + + const person = (id, dn, extra = {}) => ({ id, displayName: dn, ...extra }); + + it('links a candidate sitting EXACTLY on the threshold', () => { + // `confidence < threshold` continues. Read as <=, a score equal to the threshold is + // rejected -- and the threshold is a slider an admin sets, so "60 or better" quietly + // becoming "better than 60" changes who links with no visible cause. + const r = rules({ linkThreshold: 60 }); + const links = buildLinks( + [person('p1', 'John Doe')], + [person('i1', 'John Doe')], + r, + ); + expect(links).toHaveLength(1); + expect(links[0].confidence).toBe(60); + }); + + it('refuses a candidate just under the threshold', () => { + const r = rules({ linkThreshold: 61 }); + expect(buildLinks([person('p1', 'John Doe')], [person('i1', 'John Doe')], r)).toHaveLength(0); + }); + + it('takes the HIGHEST scoring candidate, not the first one seen', () => { + // Candidates are scored in insertion order, so the better match is listed SECOND on + // purpose: without the > comparison the first plausible identity wins and the person + // is linked to the weaker match. + const r = rules(); + const links = buildLinks( + [person('p1', 'John Doe', { employeeId: 'E1' })], + [person('i-weak', 'John Doe'), person('i-strong', 'John Doe', { employeeId: 'E1' })], + r, + ); + expect(links).toHaveLength(1); + expect(links[0].identityId).toBe('i-strong'); + expect(links[0].confidence).toBe(100); // 95 + 60, clamped by the cap + }); + + it('refuses an account type outside onlyLinkTypes', () => { + // Service and Shared accounts are deliberately left unlinked. Read as "always link", + // a shared mailbox is attached to whichever person its name resembles. + const r = rules({ + onlyLinkTypes: ['Secondary'], + accountTypeRules: [{ accountType: 'Service', priority: 1, patterns: ['^svc-'] }], + }); + expect(buildLinks([person('p1', 'svc-john doe')], [person('i1', 'John Doe')], r)).toHaveLength(0); + }); +}); + +describe('buildLinks - the ambiguity guard', () => { + const nameOnlyRules = { + signals: [ + { name: 'employeeId', type: 'exact', field: 'employeeId', weight: 95 }, + { name: 'fullName', type: 'name', level: 'full', weight: 60 }, + ], + linkThreshold: 50, + onlyLinkTypes: ['Secondary'], + accountTypeRules: [], + }; + + it('refuses a NAME-ONLY match that ties across two identities', () => { + // Two different people who genuinely share a name. Linking either one merges two + // humans, and the tie means there is no evidence to prefer one -- so the safe answer + // is to leave it orphan for review. This guard is the difference between "we do not + // know" and a confident wrong answer. + const links = buildLinks( + [{ id: 'p1', displayName: 'John Doe' }], + [{ id: 'i1', displayName: 'John Doe' }, { id: 'i2', displayName: 'John Doe' }], + nameOnlyRules, + ); + expect(links).toHaveLength(0); + }); + + it('links a NAME-ONLY match when there is exactly one candidate', () => { + // The paired case: the guard must fire on ambiguity, not on name matches generally. + const links = buildLinks( + [{ id: 'p1', displayName: 'John Doe' }], + [{ id: 'i1', displayName: 'John Doe' }], + nameOnlyRules, + ); + expect(links).toHaveLength(1); + expect(links[0].identityId).toBe('i1'); + }); + + it('links a tie that rests on a STRONG signal, not just a name', () => { + // Same tie, but both candidates share the employee id -- that is a deliberate data + // duplicate, not two people who happen to share a name, so the guard must not fire. + const links = buildLinks( + [{ id: 'p1', displayName: 'John Doe', employeeId: 'E1' }], + [ + { id: 'i1', displayName: 'John Doe', employeeId: 'E1' }, + { id: 'i2', displayName: 'John Doe', employeeId: 'E1' }, + ], + nameOnlyRules, + ); + expect(links).toHaveLength(1); + }); + + it('links when nothing matched at all only if above threshold - otherwise stays orphan', () => { + expect(buildLinks( + [{ id: 'p1', displayName: 'Nobody Here' }], + [{ id: 'i1', displayName: 'Someone Else' }], + nameOnlyRules, + )).toHaveLength(0); + }); +}); diff --git a/test/unit/EntraIDCrawlerPhases.Tests.ps1 b/test/unit/EntraIDCrawlerPhases.Tests.ps1 index cf738eabd..56e457ade 100644 --- a/test/unit/EntraIDCrawlerPhases.Tests.ps1 +++ b/test/unit/EntraIDCrawlerPhases.Tests.ps1 @@ -631,6 +631,28 @@ Describe 'Add-EntraAppRoleAssignment' { Describe 'Expand-EntraAppRoleGroupAssignments' { BeforeEach { Reset-PhaseTestState } + It 'announces the expansion when there is at least one group, and stays quiet otherwise' { + # `groupCount -gt 0` guards the only line telling an operator that a /transitiveMembers + # fan-out is starting -- the slowest part of the phase. Read as -gt 1, a run expanding + # exactly ONE group looks like it stalled with no explanation; read as always, an + # empty run claims to be expanding nothing. + $script:said = [System.Collections.Generic.List[string]]::new() + Mock Write-Host { $script:said.Add([string]$Object) } + Mock Update-CrawlerProgress -MockWith { } + Mock Invoke-FGGetRequest -ParameterFilter { $URI -match 'transitiveMembers' } -MockWith { + @([pscustomobject]@{ id = 'u1'; '@odata.type' = '#microsoft.graph.user' }) + } + + $one = @{ 'grp1' = [System.Collections.Generic.List[object]]::new() } + $one['grp1'].Add(@{ roleResId = 'rr1'; roleId = 'role-a'; sourceAssignmentId = 'aa1'; appName = 'App One' }) + Expand-EntraAppRoleGroupAssignments -GroupAssns $one | Out-Null + ($script:said -join "`n") | Should -Match 'Expanding 1 group' + + $script:said.Clear() + Expand-EntraAppRoleGroupAssignments -GroupAssns @{} | Out-Null + ($script:said -join "`n") | Should -Not -Match 'Expanding' + } + It 'fans a group role assignment out to one row per transitive user member' { Mock Invoke-FGGetRequest -ParameterFilter { $URI -match 'transitiveMembers' } -MockWith { @( @@ -1165,12 +1187,18 @@ Describe 'Get-EntraServicePrincipalData' { Mock Invoke-FGGetDeltaRequest -MockWith { @{ value = @( [pscustomobject]@{ id = 'sp1'; displayName = 'Changed' } + [pscustomobject]@{ id = 'sp3'; displayName = 'Also changed' } [pscustomobject]@{ id = 'sp2'; '@removed' = [pscustomobject]@{ reason = 'deleted' } } ); deltaToken = 'next-tok' } } $r = Get-EntraServicePrincipalData -SystemId 5 -SyncMode 'delta' $r.spDeltaHit | Should -BeTrue - @($r.sps).Count | Should -Be 1 + # Assert WHICH service principals came back, not just how many. With one live and + # one removed the counts are symmetric, so dropping the negation swaps the two + # lists and every assertion still reads 1 -- the crawler would then upsert the + # deleted SP and tombstone the live ones. + @($r.sps).Count | Should -Be 2 + @($r.sps.id) | Should -Be @('sp1', 'sp3') @($r.removedSpIds) | Should -Be @('sp2') $r.newSpsToken | Should -Be 'next-tok' } @@ -1289,17 +1317,21 @@ Describe 'Sync-EntraSignInLogs' { Mock Get-Date -MockWith { [datetime]::SpecifyKind([datetime]'2026-01-10T00:00:00', 'Utc') } $script:said = [System.Collections.Generic.List[string]]::new() Mock Write-Host { $script:said.Add([string]$Object) } + # ASYMMETRIC on purpose: TWO events that aggregate against ONE that cannot. With + # one of each, inverting the negation swaps two 1s and the message reads the same. Mock Invoke-FGGetRequestStream -MockWith { @( [pscustomobject]@{ userId = 'u1'; appId = 'a1'; createdDateTime = '2026-01-09T10:00:00Z'; status = [pscustomobject]@{ errorCode = 0 } } - [pscustomobject]@{ userId = 'u2'; appId = 'unknown'; createdDateTime = '2026-01-09T11:00:00Z'; status = [pscustomobject]@{ errorCode = 0 } } + [pscustomobject]@{ userId = 'u2'; appId = 'a1'; createdDateTime = '2026-01-09T10:30:00Z'; status = [pscustomobject]@{ errorCode = 0 } } + [pscustomobject]@{ userId = 'u3'; appId = 'unknown'; createdDateTime = '2026-01-09T11:00:00Z'; status = [pscustomobject]@{ errorCode = 0 } } ) } Sync-EntraSignInLogs -SystemId 5 -Sps @([pscustomobject]@{ id = 'sp1'; appId = 'a1' }) -SignInLogsDays 1 -Timings ([ordered]@{}) ($script:said -join "`n") | Should -Match 'Skipped 1 events' - (Get-Sent { $_.Endpoint -eq 'ingest/principal-activity' })[0].Records.Count | Should -Be 1 + # ...and the two that DID aggregate are uploaded, as two distinct (user, app) pairs. + (Get-Sent { $_.Endpoint -eq 'ingest/principal-activity' })[0].Records.Count | Should -Be 2 } It 'records a PARTIAL slice failure and still uploads the slice that worked' { diff --git a/test/unit/MidpointCrawlerPhases.Tests.ps1 b/test/unit/MidpointCrawlerPhases.Tests.ps1 index c7a3355d1..8307c4520 100644 --- a/test/unit/MidpointCrawlerPhases.Tests.ps1 +++ b/test/unit/MidpointCrawlerPhases.Tests.ps1 @@ -88,6 +88,24 @@ Describe 'Sync-MidpointSystems' { $script:phaseErrors.Count | Should -Be 0 } + It 'registers midPoint itself as enabled and sync-enabled' { + # midPoint is the system being crawled, so unlike its connected resources it MUST be + # syncable -- registered $false, the tenant is onboarded once and never refreshed. + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'resources' } -MockWith { @() } + Mock Invoke-MidpointSearchStream -MockWith { 0 } + $script:mpSysRecs = [System.Collections.Generic.List[object]]::new() + Mock Invoke-IngestAPI -MockWith { foreach ($r in @($Body.records)) { $script:mpSysRecs.Add($r) }; @{} } + Mock Invoke-RestMethod -MockWith { @([pscustomobject]@{ systemType = 'Midpoint'; tenantId = 'https://mp.example.com'; id = 10 }) } + + Sync-MidpointSystems -RestRoot 'https://mp.example.com' -ApiBaseUrl 'https://x/api' -ApiKey 'k' | Out-Null + + $mp = @($script:mpSysRecs | Where-Object { $_.tenantId -eq 'https://mp.example.com' }) + $mp | Should -HaveCount 1 + $mp[0].systemType | Should -Be 'Midpoint' + $mp[0].enabled | Should -BeTrue + $mp[0].syncEnabled | Should -BeTrue + } + It 'throws (critical phase) when the system id cannot be resolved' { Mock Invoke-MidpointSearch -MockWith { @() } Mock Invoke-MidpointSearchStream -MockWith { 0 } @@ -270,6 +288,60 @@ Describe 'Sync-MidpointResources' { $script:phaseErrors.Count | Should -Be 0 } + It 'falls back to roleType when a role carries no subtype' { + # `if ($subs.Count -eq 0) { $subs = ... roleType }`. midPoint tenants classify roles + # either way round; without the fallback every role that uses roleType instead of + # subtype silently lands on the default resourceType, so a whole tenant's roles are + # mis-typed with nothing to indicate it. + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'roles' } -MockWith { + @([pscustomobject]@{ oid = 'role-1'; name = 'app-owner'; displayName = 'App Owner'; roleType = 'application' }) + } + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'services' } -MockWith { @() } + $mapping = @([pscustomobject]@{ archetype = ''; subtype = 'application'; resourceType = 'Application' }) + + $r = Sync-MidpointResources -MidpointSystemId 10 -ArchetypeMapping $mapping + + $r.resourceOidToType['role-1'] | Should -Be 'Application' + } + + It 'prefers subtype over roleType when both are present' { + # The paired case, so "always use roleType" cannot pass the test above. + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'roles' } -MockWith { + @([pscustomobject]@{ oid = 'role-1'; name = 'app-owner'; displayName = 'App Owner' + subtype = 'business'; roleType = 'application' }) + } + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'services' } -MockWith { @() } + $mapping = @( + [pscustomobject]@{ archetype = ''; subtype = 'business'; resourceType = 'BusinessRole' } + [pscustomobject]@{ archetype = ''; subtype = 'application'; resourceType = 'Application' } + ) + + $r = Sync-MidpointResources -MidpointSystemId 10 -ArchetypeMapping $mapping + + $r.resourceOidToType['role-1'] | Should -Be 'BusinessRole' + } + + It 'fetches the archetype catalog only when a mapping row keys on an archetype' { + # `@(mapping | where archetype).Count -gt 0` decides whether an extra midPoint round + # trip happens at all. Read as "always", every tenant pays for a catalog fetch it + # does not use; read as "never", archetype-keyed mappings silently stop resolving + # and every role falls back to the default resourceType. + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'roles' } -MockWith { + @([pscustomobject]@{ oid = 'role-1'; name = 'admin'; displayName = 'Administrator' }) + } + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'services' } -MockWith { @() } + Mock Get-MidpointArchetypeLabels -MockWith { @{} } + + # No archetype key anywhere in the mapping -> no catalog fetch. + Sync-MidpointResources -MidpointSystemId 10 -ArchetypeMapping (ConvertTo-MapRows $null @('archetype','subtype','resourceType')) | Out-Null + Should -Invoke Get-MidpointArchetypeLabels -Exactly 0 + + # One row that DOES key on an archetype -> exactly one fetch. + $withArch = @([pscustomobject]@{ archetype = 'Business Role'; subtype = ''; resourceType = 'BusinessRole' }) + Sync-MidpointResources -MidpointSystemId 10 -ArchetypeMapping $withArch | Out-Null + Should -Invoke Get-MidpointArchetypeLabels -Exactly 1 + } + It 'records a Roles phase error when the roles fetch throws (services still run)' { Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'roles' } -MockWith { throw 'roles 500' } Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'services' } -MockWith { @() } @@ -346,6 +418,12 @@ Describe 'Add-MidpointShadowPage' { $syncedRes.Contains('ent-sh') | Should -BeTrue $skipped.generic | Should -Be 1 ($byDn.Values) | Should -Contain 'ent-sh' + # The res-unsynced shadow must produce NO bucket at all. Asserting only on + # $acct[11] cannot see it: the stray record lands under a different key (the + # missing system id) and the count that was checked stays 1 either way. What + # that costs in production is a shadow filed against a system Identity Atlas + # never registered. + @($acct.Keys) | Should -Be @(11) } } @@ -494,6 +572,28 @@ Describe 'Sync-MidpointReviews' { $script:phaseErrors.Count | Should -Be 0 } + It 'skips a review case that names only one side of the pair' { + # `-not principalOid -or -not targetOid`. A case missing either side cannot become a + # certification decision: as -and, only a case missing BOTH is dropped, so a review + # of "somebody, on role-1" or "u-1, on nothing" is recorded as a real decision with + # half of it blank -- and certification records are what an auditor reads. + Mock Invoke-MidpointSearch -ParameterFilter { $Type -eq 'accessCertificationCampaigns' } -MockWith { + @([pscustomobject]@{ oid = 'camp-1'; name = 'Q1 review'; state = 'closed' + case = @( + [pscustomobject]@{ '@id' = '1'; objectRef = $null; targetRef = [pscustomobject]@{ oid = 'role-1'; type = 'c:RoleType' }; outcome = 'accept' } + [pscustomobject]@{ '@id' = '2'; objectRef = @{ oid = 'u-1' }; targetRef = $null; outcome = 'accept' } + [pscustomobject]@{ '@id' = '3'; objectRef = @{ oid = 'u-1' }; targetRef = [pscustomobject]@{ oid = 'role-1'; type = 'c:RoleType' }; outcome = 'accept' } + ) }) + } + + Sync-MidpointReviews -MidpointSystemId 10 -SyncedResourceIds (New-StrSet 'role-1') -UserOidToName @{ 'u-1' = 'Alice' } + + # Only the complete case survives. + $sent = Get-Sent { $_.Endpoint -eq 'ingest/governance/certifications' } + $sent[0].Records.Count | Should -Be 1 + $script:phaseErrors.Count | Should -Be 0 + } + It 'records a phase error when the campaign fetch throws' { Mock Invoke-MidpointSearch -MockWith { throw 'campaigns 500' } Sync-MidpointReviews -MidpointSystemId 10 -SyncedResourceIds (New-StrSet 'x') -UserOidToName @{} diff --git a/test/unit/OmadaCrawlerPhases.Tests.ps1 b/test/unit/OmadaCrawlerPhases.Tests.ps1 index 9889ed7b4..0f3c19f7b 100644 --- a/test/unit/OmadaCrawlerPhases.Tests.ps1 +++ b/test/unit/OmadaCrawlerPhases.Tests.ps1 @@ -218,6 +218,23 @@ Describe 'Sync-OmadaContexts' { Describe 'Sync-OmadaIdentities' { BeforeEach { Reset-PhaseTestState; Mock Send-IngestBatch -MockWith $script:SendMock } + It 'drops an identity that has an id but no display name' { + # Same `externalId -and displayName` guard as the accounts phase. As -or, a person + # row with no name is written to the Identities table, and an identity nobody can + # recognise is worse than one that is missing: it looks like a real person. + Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/Identity' } -MockWith { + @( + [pscustomobject]@{ UId = 'id-1'; IDENTITYID = 'ID-1'; FIRSTNAME = 'Alice'; LASTNAME = 'Smith'; IDENTITYTYPE = [pscustomobject]@{ Value = 'Employee' } } + [pscustomobject]@{ UId = 'id-2'; IDENTITYID = 'ID-2'; IDENTITYTYPE = [pscustomobject]@{ Value = 'Employee' } } # no names + ) + } + + $r = Sync-OmadaIdentities -SystemId 1 -IdentityTypesForIdentityTable @('Employee') + + (Get-Sent { $_.Endpoint -eq 'ingest/identities' })[0].Records.Count | Should -Be 1 + (Get-Sent { $_.Endpoint -eq 'ingest/identities' })[0].Records[0].externalId | Should -Be 'id-1' + } + It 'ingests person-type identities and returns the lookup + in-table set' { Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/Identity' } -MockWith { @( @@ -282,6 +299,28 @@ Describe 'Sync-OmadaAccounts' { $script:phaseErrors.Count | Should -Be 0 } + It 'drops an account that has an id but no display name' { + # `externalId -and displayName`. Read as -or, a record with one half missing is + # ingested anyway: a principal row with a blank name, which is what a reviewer then + # sees in the matrix and cannot identify. The same filter guards contexts and + # identities; this is the account one. + Mock Invoke-ODataPagedRequest -ParameterFilter { $Path -eq '/User' } -MockWith { + @( + [pscustomobject]@{ UId = 'acc-1'; UserName = 'alice'; FIRSTNAME = 'Alice'; LASTNAME = 'Smith'; IDENTITYREF = [pscustomobject]@{ IDENTITYID = 'ID-1' } } + # No names at all -> displayName resolves empty. Real: service accounts + # created by an integration routinely carry no person name. + [pscustomobject]@{ UId = 'acc-2'; UserName = 'svc'; IDENTITYREF = [pscustomobject]@{ IDENTITYID = 'ID-1' } } + ) + } + $lookup = @{ 'ID-1' = @{ uid = 'id-1'; identityType = 'Employee' } } + + $r = Sync-OmadaAccounts -SystemId 4 -IdentityLookup $lookup + + (Get-Sent { $_.Scope.principalType -eq 'User' })[0].Records.Count | Should -Be 1 + (Get-Sent { $_.Scope.principalType -eq 'User' })[0].Records[0].externalId | Should -Be 'acc-1' + @($r.allAccounts).Count | Should -Be 2 # both fetched; only one ingestable + } + It 'uploads a principal whose type is none of the three built-in ones' { # Accounts are ingested in three fixed buckets (User / ExternalUser / # ServicePrincipal) plus a catch-all for anything an operator's typeMappings @@ -605,6 +644,70 @@ Describe 'Omada config resolution' { $c.SyncMode | Should -Be 'full' } + It 'Send-OmadaResourceBatch attributes the MAIN key to the Omada system itself' { + # The '__main__' key means "Omada Identity", everything else is a connected system + # looked up in the map. Read as -ne, the two swap: Omada's own resources are filed + # under a connected system's id (or none at all) and every connected system's + # resources are filed under Omada. + $script:said = [System.Collections.Generic.List[string]]::new() + Mock Write-Host { $script:said.Add([string]$Object) } + Mock Send-IngestBatch -MockWith { @{ inserted = 1; updated = 0; deleted = 0 } } + + Send-OmadaResourceBatch -Key '__main__' -Records @(@{ id = 'r1' }) -SystemId 7 ` + -OmadaSystemMap @{ 'conn-uid' = 9 } -AllOmadaSystems @([pscustomobject]@{ UId = 'conn-uid'; DisplayName = 'AD' }) | Out-Null + + Should -Invoke Send-IngestBatch -Exactly 1 -ParameterFilter { $SystemId -eq 7 } + ($script:said -join "`n") | Should -Match 'Resources \(Omada,' + } + + It 'Send-OmadaResourceBatch attributes a connected-system key to that system' { + # The paired case, so "always take the main branch" cannot pass both. + $script:said = [System.Collections.Generic.List[string]]::new() + Mock Write-Host { $script:said.Add([string]$Object) } + Mock Send-IngestBatch -MockWith { @{ inserted = 1; updated = 0; deleted = 0 } } + + Send-OmadaResourceBatch -Key 'conn-uid' -Records @(@{ id = 'r1' }) -SystemId 7 ` + -OmadaSystemMap @{ 'conn-uid' = 9 } -AllOmadaSystems @([pscustomobject]@{ UId = 'conn-uid'; DisplayName = 'AD' }) | Out-Null + + Should -Invoke Send-IngestBatch -Exactly 1 -ParameterFilter { $SystemId -eq 9 } + ($script:said -join "`n") | Should -Match 'Resources \(AD,' + } + + It 'Get-OmadaContextMembersFromIdentityFields drops a reference to an unsynced context' { + # `-not ContextUid -or -not SyncedContextIds.Contains(ContextUid)`. An identity can + # reference an org unit that was filtered out of this run -- the reference exists, + # the context does not. Read as -and, that produces a membership row pointing at a + # context id Identity Atlas never received, which is a dangling row nothing cleans up. + $synced = [System.Collections.Generic.HashSet[string]]::new() + [void]$synced.Add('ctx-known') + $inTable = [System.Collections.Generic.HashSet[string]]::new() + [void]$inTable.Add('id-1') + + $rows = Get-OmadaContextMembersFromIdentityFields ` + -AllIdentities @([pscustomobject]@{ UId = 'id-1'; OUREF = [pscustomobject]@{ UId = 'ctx-gone' } }) ` + -ContextObjectTypes @([pscustomobject]@{ identityField = 'OUREF' }) ` + -SyncedContextIds $synced -IdentityUidInIdentitiesTable $inTable + + @($rows) | Should -HaveCount 0 + } + + It 'Get-OmadaContextMembersFromIdentityFields keeps a reference to a synced context' { + # The paired case, so "never emit anything" cannot pass the test above. + $synced = [System.Collections.Generic.HashSet[string]]::new() + [void]$synced.Add('ctx-known') + $inTable = [System.Collections.Generic.HashSet[string]]::new() + [void]$inTable.Add('id-1') + + $rows = Get-OmadaContextMembersFromIdentityFields ` + -AllIdentities @([pscustomobject]@{ UId = 'id-1'; OUREF = [pscustomobject]@{ UId = 'ctx-known' } }) ` + -ContextObjectTypes @([pscustomobject]@{ identityField = 'OUREF' }) ` + -SyncedContextIds $synced -IdentityUidInIdentitiesTable $inTable + + @($rows) | Should -HaveCount 1 + $rows[0].contextId | Should -Be 'ctx-known' + $rows[0].memberId | Should -Be 'id-1' + } + It 'Resolve-OmadaSyncToggles turns every phase ON when nothing is configured' { # The test above pins three of the nine toggles, two of which it overrides # to false -- so six defaults were unasserted. Every one of them defaults @@ -720,6 +823,33 @@ Describe 'Omada setup helpers' { Should -Invoke Connect-ODataAPI -Exactly 1 } + It 'Write-OmadaPhaseLine marks a failed phase FAILED, in red' { + # Status and colour are two separate `-eq 'ok'` reads on the same value. Flip + # either and the end-of-run summary lies about the run: a failed phase printed as + # "ok" in green is the one line an operator scans to decide whether to look further. + $script:said = [System.Collections.Generic.List[string]]::new() + $script:colours = [System.Collections.Generic.List[string]]::new() + Mock Write-Host { $script:said.Add([string]$Object); $script:colours.Add([string]$ForegroundColor) } + + Write-OmadaPhaseLine -Phase ([pscustomobject]@{ name = 'Resources'; status = 'error'; durationMs = 12 }) + + ($script:said -join '') | Should -Match 'FAILED' + $script:colours | Should -Contain 'Red' + } + + It 'Write-OmadaPhaseLine marks a successful phase ok, in green' { + # The paired case: without it, hard-coding either read to the failure branch passes. + $script:said = [System.Collections.Generic.List[string]]::new() + $script:colours = [System.Collections.Generic.List[string]]::new() + Mock Write-Host { $script:said.Add([string]$Object); $script:colours.Add([string]$ForegroundColor) } + + Write-OmadaPhaseLine -Phase ([pscustomobject]@{ name = 'Resources'; status = 'ok'; durationMs = 12 }) + + ($script:said -join '') | Should -Match 'ok' + ($script:said -join '') | Should -Not -Match 'FAILED' + $script:colours | Should -Contain 'Green' + } + It 'Send-OmadaPhaseResults posts nothing without a real job id' { # Guard is `JobId -le 0`. Read as `-lt 0`, a run with no job (id 0) POSTs to # /crawlers/jobs/0/phases -- an endpoint for a job that does not exist. From ba7a4a582040774efbb2e9679adcfe9da02bb240 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 13:25:45 +0200 Subject: [PATCH 04/15] ci: run Stryker weekly and publish the JS fault-detection score Mutation testing existed on the JS side but ran nowhere -- four Stryker configs, all `break: null`, invoked by hand. Nothing re-ran them, so a test that stopped discriminating would have gone unnoticed indefinitely, and the coverage page's Mutation column read "-" for API and UI. js-mutation.yml mirrors ps-mutation.yml: weekly Monday 05:00 UTC and on demand, never on a pull request (a run this heavy can never be a required check -- GitHub leaves one that never reports permanently pending, so on PRs it would only LOOK like a gate). One job per scope, fail-fast off, so a regression in one names itself in the check list without costing the other three their measurement. Each config now carries an enforced floor a few points under its measured score, verified green by running all four: auth 100.00 >= 98, effective access 94.65 >= 92, account linking 81.94 >= 80, UI 93.79 >= 91. That run also settled which score `break` compares against -- the TOTAL column (81.94), not "covered" (85.43) -- so the merged report's arithmetic counts NoCoverage as missed, exactly as the gate does. The publish job merges each package's per-scope reports into the shape generate-coverage-doc.py already reads and commits the refreshed page. Which reports it demands is derived from the configs' own jsonReporter.fileName, so a fifth scope requires its report with no edit; a missing one is a hard error rather than a quietly smaller merge, which would publish a score measured over less code than the page's scope note claims. It publishes even when a scope is below its floor: freezing the page at the last good number would leave it asserting something false for as long as the regression lasted. Fixes a real bug this surfaced: the page read ONE mutation-scope declaration and applied it to every suite, so wiring JS in would have reported PowerShell's 112 files as the scope of a JS score. Scope is now resolved per suite, with a regression test that fails against the old behaviour. Also: the coverage-docs tooling had tests that CI never ran (they are only exercised by post-merge workflows), so a PR could break the page and nothing would say so until it had landed; they now run in the ci-scripts job. The gate-wiring assertion that mutation stays off pull requests is derived over *-mutation.yml rather than naming ps-mutation.yml, with a check that the glob still matches both -- a rename would otherwise pass over an empty list. Documentation corrected where this made it wrong: the recorded PowerShell floor (80 -> 85), "nothing in this repo is declared yet" (there are 127), and a status section describing the phase layers as unmeasured after they had been measured. --- .github/workflows/coverage.yml | 10 +- .github/workflows/js-mutation.yml | 190 +++++++++++++++ .github/workflows/pr.yml | 12 + .github/workflows/ps-mutation.yml | 9 +- app/api/package.json | 5 +- app/api/stryker.accountlinking.config.json | 4 +- app/api/stryker.auth.config.json | 4 +- app/api/stryker.effectiveaccess.config.json | 4 +- app/ui/package.json | 3 +- app/ui/stryker.pilot.config.json | 4 +- changes/phases-into-gate.md | 3 + .../contributing/writing-tests-that-assert.md | 69 ++++-- test/ci-scripts/test-gate-wiring.sh | 19 +- tools/generate-coverage-doc.py | 85 ++++++- tools/mutation/stryker_to_mutation_json.py | 176 ++++++++++++++ .../mutation/test_stryker_to_mutation_json.py | 217 ++++++++++++++++++ tools/test_generate_coverage_doc.py | 94 ++++++++ 17 files changed, 861 insertions(+), 47 deletions(-) create mode 100644 .github/workflows/js-mutation.yml create mode 100644 tools/mutation/stryker_to_mutation_json.py create mode 100644 tools/mutation/test_stryker_to_mutation_json.py diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 488fff269..67e263170 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -12,8 +12,9 @@ # tools/complexity/ratchet.py --emit-complexity-json (ESLint + sonarjs) for # the API/UI suites. The page surfaces it as avg/max cyclomatic + cognitive # columns. (Mutation is the heavier signal, so it is NOT run here: the weekly -# ps-mutation.yml refreshes docs/coverage/powershell/mutation.json, which -# this page just reads back — see that workflow.) +# ps-mutation.yml refreshes docs/coverage/powershell/mutation.json and the +# weekly js-mutation.yml refreshes the api/ui ones, which this page just reads +# back — see those workflows.) # 4. Renders the curated docs page (docs/reference/coverage.md) from those # summaries (tools/generate-coverage-doc.py). # 5. Commits docs/reference/coverage.md + docs/coverage/** back to main. @@ -187,8 +188,9 @@ jobs: # complexity.yml. Emits the per-unit {file,unit,line,cc,cog} array; # generate-coverage-doc.py aggregates it to avg/max per suite. Cheap (a few # seconds), so it refreshes every merge. Mutation is deliberately NOT run - # here — the weekly ps-mutation.yml owns docs/coverage/powershell/mutation.json, - # which the render step below reads back. continue-on-error keeps a + # here — the weekly ps-mutation.yml owns docs/coverage/powershell/mutation.json + # and js-mutation.yml owns docs/coverage/{api,ui}/mutation.json, all of which + # the render step below reads back. continue-on-error keeps a # measurement hiccup from blocking the docs commit, matching the suites above. - name: PowerShell complexity shell: pwsh diff --git a/.github/workflows/js-mutation.yml b/.github/workflows/js-mutation.yml new file mode 100644 index 000000000..b668a3f6a --- /dev/null +++ b/.github/workflows/js-mutation.yml @@ -0,0 +1,190 @@ +# ─── JavaScript mutation testing ───────────────────────────────────────────── +# The JS counterpart to ps-mutation.yml. Runs the committed Stryker scopes against +# their Vitest suites -- the metric that proves the tests would CATCH a bug, not +# just execute the line. Line coverage cannot tell a real assertion from an empty +# one, and this repo has repeatedly measured the gap: classifier.js sat at 97% line +# / 68% mutation, effectiveAccess/engine.js at 93% / 69%, matrixFilter.js at 99% / +# 85%. Those files were "covered" the whole time. +# +# ENFORCED. Each stryker.*.config.json carries a `thresholds.break` a few points +# under its measured score; Stryker exits non-zero below it and the step fails. +# The floors are a RATCHET -- raise them as scores rise, never lower one to make a +# red run green. A drop means a test stopped discriminating. +# +# WEEKLY AND ON DEMAND, NEVER ON A PULL REQUEST. Mutation testing re-runs the test +# suite once per mutant; it is orders of magnitude heavier than the unit run. A +# path-filtered PR workflow also can never be a required check -- GitHub leaves a +# required check that never reports permanently pending -- so on PRs it would only +# ever LOOK like a gate. A regression is caught by the Monday run, not at review +# time; that is the deliberate trade for the runtime, same as ps-mutation.yml. +# test/ci-scripts/test-gate-wiring.sh enforces this for every *-mutation.yml. +# +# SCOPE IS SMALL AND THAT IS THE POINT OF PUBLISHING IT. Ten API/UI files are +# mutation-tested out of ~410 eligible; the remaining backlog is counted in +# .ci/js-mutation-scope-baseline.json and guarded by app/api/src/mutationScope. +# guard.test.js, which fails when new code is added without a decision. Publishing +# the score to the coverage docs page carries the scope note with it ("covers 10 +# file(s) of 189 -- 5% of the suite's coverable lines"), so the number can never be +# mistaken for a suite-wide one. A quiet 88% would have been the more flattering +# option and the dishonest one. +# +# Like ps-mutation.yml, this owns the Mutation column for its suites on the Test +# Coverage docs page: the publish job merges each package's per-scope reports into +# docs/coverage/{api,ui}/mutation.json, re-renders docs/reference/coverage.md from +# the already-committed coverage summaries, and commits both back to main. +# coverage.yml (every merge) just reads those files back -- it never runs mutation. +# ───────────────────────────────────────────────────────────────────────────── +name: JavaScript mutation testing + +on: + schedule: + - cron: '0 5 * * 1' # Weekly, Monday 05:00 UTC (after ps-mutation.yml at 03:00, + # so the two never contend for the same docs commit) + workflow_dispatch: + # Deliberately NOT on pull_request -- see the header. + +concurrency: + group: js-mutation-${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +jobs: + # ── Measure each scope, in parallel ───────────────────────────────────────── + # One job per Stryker config rather than one job running all four: the check + # names then say WHICH scope regressed without opening a log, and wall-clock is + # the slowest scope instead of their sum. + mutation: + name: 'Mutation: ${{ matrix.scope.label }}' + runs-on: ubuntu-latest + timeout-minutes: 45 + strategy: + # A scope that lands below its floor must not cancel the other three. The + # point of the weekly run is a complete picture; losing three numbers to one + # regression would also leave the publish job with nothing to merge. + fail-fast: false + matrix: + scope: + - { pkg: api, name: auth, label: 'API auth' } + - { pkg: api, name: effectiveaccess, label: 'API effective access' } + - { pkg: api, name: accountlinking, label: 'API account linking' } + - { pkg: ui, name: pilot, label: 'UI' } + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + + - name: Setup Node.js + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v6 + with: + node-version: '24' + cache: 'npm' + cache-dependency-path: app/${{ matrix.scope.pkg }}/package-lock.json + + - name: Install dependencies + working-directory: app/${{ matrix.scope.pkg }} + run: npm ci + + - name: Run mutation testing (enforced — fails below the config's break floor) + working-directory: app/${{ matrix.scope.pkg }} + run: npm run test:mutation:${{ matrix.scope.name }} + + # if: always() so a scope that fell below its floor still hands its report to + # the publish job. The failing number is the one most worth reading, and the + # docs page has to be able to state it. + - name: Upload mutation report + if: always() + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: stryker-${{ matrix.scope.pkg }}-${{ matrix.scope.name }} + path: app/${{ matrix.scope.pkg }}/reports/stryker-*.json + if-no-files-found: error + + # ── Publish the scores to the coverage docs ───────────────────────────────── + publish: + name: Publish mutation scores + needs: [mutation] + # always(): a scope below its floor still publishes its real number. Freezing + # the page at the last good score would leave it asserting something false for + # as long as the regression lasted -- the measure jobs above carry the red. + # A scope that produced NO report (a runner or install failure) is different: + # the merge below refuses to publish a partial picture and fails here too. + if: always() + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: write # commits the refreshed scores to main + steps: + # GITHUB_TOKEN cannot push to protected main, so the bot token is required. + - name: Generate bot token + id: bot-token + uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 + with: + client-id: ${{ secrets.BOT_APP_ID }} + private-key: ${{ secrets.BOT_PRIVATE_KEY }} + + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + token: ${{ steps.bot-token.outputs.token || github.token }} + + - name: Download API mutation reports + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + pattern: stryker-api-* + merge-multiple: true + path: app/api/reports + + - name: Download UI mutation reports + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + pattern: stryker-ui-* + merge-multiple: true + path: app/ui/reports + + # Which reports must be present is derived from the Stryker configs + # themselves, so adding a fifth scope requires its report with no edit here. + # A missing one is a hard error: a merge that silently dropped a scope would + # publish a score measured over less code than the page's scope note claims. + - name: Merge per-scope reports into the docs inputs + run: | + python3 tools/mutation/stryker_to_mutation_json.py \ + --package-dir app/api --out docs/coverage/api/mutation.json + python3 tools/mutation/stryker_to_mutation_json.py \ + --package-dir app/ui --out docs/coverage/ui/mutation.json + + # Re-renders the page from the ALREADY-COMMITTED per-suite coverage summaries + # plus the fresh mutation.json files, so line/complexity numbers stay as-of + # the last coverage.yml run and only the mutation columns move. + - name: Re-render the coverage docs page + env: + GITHUB_SHA: ${{ github.sha }} + run: | + python3 tools/generate-coverage-doc.py \ + --coverage-dir docs/coverage \ + --out docs/reference/coverage.md \ + --commit "$GITHUB_SHA" + + - name: Commit mutation scores + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git add docs/coverage/api/mutation.json docs/coverage/ui/mutation.json docs/reference/coverage.md + if git diff --cached --quiet; then + echo "Mutation scores unchanged — nothing to commit" + exit 0 + fi + git commit -m "docs: refresh JavaScript mutation scores" + # coverage.yml / bump-version.yml / ps-mutation.yml may push to main + # concurrently — rebase & retry rather than lose the update. + for attempt in 1 2 3 4 5; do + if git push; then + echo "Pushed on attempt $attempt" + exit 0 + fi + echo "Push failed (attempt $attempt) — rebasing onto latest main" + git pull --rebase origin main + done + echo "::error::Could not push mutation update after 5 attempts" + exit 1 diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 227163b9f..408325a54 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -912,6 +912,18 @@ jobs: bash test/ci-scripts/test-dor-push-lease.sh bash test/ci-scripts/test-gate-wiring.sh + # The two Python tools that render and feed the Test Coverage docs page. + # Both are run only by post-merge workflows (coverage.yml, js-mutation.yml), + # so without this step a PR could break the page and nothing would say so + # until it had already landed on main — which is how these tests came to + # exist and never run. The ratchet gates test their own logic in their own + # jobs; these have no gate job to live in, so they live here. + - name: Test the coverage-docs tooling + run: | + python3 -m pip install --quiet pytest + python3 -m pytest tools/test_generate_coverage_doc.py \ + tools/mutation/test_stryker_to_mutation_json.py -q + # ── Stage 15: Docs site build (strict) ──────────────────────────────────── # `docs.yml` publishes the site, but it only runs on push to main — so until # now a PR could break the nav, an internal cross-link or a link to a source diff --git a/.github/workflows/ps-mutation.yml b/.github/workflows/ps-mutation.yml index 2aa10efbf..b80cfc249 100644 --- a/.github/workflows/ps-mutation.yml +++ b/.github/workflows/ps-mutation.yml @@ -3,14 +3,15 @@ # transforms (the pure ConvertTo-* record-shapers) against their Pester suites -- # the metric that proves the tests would CATCH a bug, not just execute the line. # -# ENFORCED (.ci/psmutant.config.json has thresholds.break = 80): the run FAILS when +# ENFORCED (.ci/psmutant.config.json has thresholds.break = 85): the run FAILS when # the mutation score drops below the floor — PSMutant returns a non-zero exit code, # which the step below rethrows. Mutation is far heavier than the unit suite, so it # runs WEEKLY and on demand only — never on a pull request. A regression is caught by # the Monday run, not at review time; that is the deliberate trade for the runtime. # -# Because it's the ONLY place mutation runs, the weekly/on-demand run also owns the -# mutation figure on the Test Coverage docs page: it writes the score to +# Because it's the only place PowerShell mutation runs, the weekly/on-demand run also +# owns the PowerShell mutation figure on the Test Coverage docs page (js-mutation.yml +# owns the API and UI ones, the same way): it writes the score to # docs/coverage/powershell/mutation.json, re-renders docs/reference/coverage.md from # the already-committed coverage summaries (tools/generate-coverage-doc.py), and # commits both back to main. coverage.yml (every merge) just reads that file back — @@ -74,7 +75,7 @@ jobs: - name: Upload mutation report if: always() - uses: actions/upload-artifact@v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 with: name: ps-mutation-report path: reports/ps-mutation.json diff --git a/app/api/package.json b/app/api/package.json index 1ffecfd9e..03e62e3b8 100644 --- a/app/api/package.json +++ b/app/api/package.json @@ -17,7 +17,10 @@ "postinstall": "patch-package || true", "build:node-launcher": "node ../desktop/scripts/build-node-launcher.mjs", "build:node-launcher:skip-ui": "node ../desktop/scripts/build-node-launcher.mjs --skip-ui-build", - "test:mutation": "stryker run stryker.auth.config.json" + "test:mutation": "npm run test:mutation:auth && npm run test:mutation:effectiveaccess && npm run test:mutation:accountlinking", + "test:mutation:auth": "stryker run stryker.auth.config.json", + "test:mutation:effectiveaccess": "stryker run stryker.effectiveaccess.config.json", + "test:mutation:accountlinking": "stryker run stryker.accountlinking.config.json" }, "devDependencies": { "@eslint/js": "^10.0.1", diff --git a/app/api/stryker.accountlinking.config.json b/app/api/stryker.accountlinking.config.json index 78cf772ad..583b4cdc1 100644 --- a/app/api/stryker.accountlinking.config.json +++ b/app/api/stryker.accountlinking.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "Mutation testing for account linking: the code that decides which accounts belong to the same PERSON. Scoped like the auth and effective-access configs -- these files plus their own unit tests, which are self-contained (no filesystem, no crawler manifests) so they run clean inside Stryker's sandbox. WHY THESE FILES: they sit at 83-99% line coverage and both failure directions are silent and serious. Link too eagerly and two people become one identity, so one of them inherits the other's access in every view Identity Atlas presents. Link too shyly and one person stays fragmented, so their true combined access is never visible to a reviewer. Neither throws, and coverage cannot tell either apart from correct behaviour -- which is the same combination that made readTokens.js and the effective-access engine worth measuring. MUTATOR SET: StringLiteral and ObjectLiteral disabled, matching the sibling configs. Same cost, restated: SQL strings stop being mutated too, so this cannot tell you a query is correct; that stays the contract tests' job. Measurement, not a gate -- `break` is null. Run: cd app/api && npx stryker run stryker.accountlinking.config.json", + "_comment": "Mutation testing for account linking: the code that decides which accounts belong to the same PERSON. Scoped like the auth and effective-access configs -- these files plus their own unit tests, which are self-contained (no filesystem, no crawler manifests) so they run clean inside Stryker's sandbox. WHY THESE FILES: they sit at 83-99% line coverage and both failure directions are silent and serious. Link too eagerly and two people become one identity, so one of them inherits the other's access in every view Identity Atlas presents. Link too shyly and one person stays fragmented, so their true combined access is never visible to a reviewer. Neither throws, and coverage cannot tell either apart from correct behaviour -- which is the same combination that made readTokens.js and the effective-access engine worth measuring. MUTATOR SET: StringLiteral and ObjectLiteral disabled, matching the sibling configs. Same cost, restated: SQL strings stop being mutated too, so this cannot tell you a query is correct; that stays the contract tests' job. Enforced: see MUTATION FLOOR below. Run: cd app/api && npm run test:mutation:accountlinking MUTATION FLOOR: `break` is set a few points under the measured score, and is a RATCHET — it only ever goes up. It is enforced by the weekly .github/workflows/js-mutation.yml (never on a PR: mutation is far heavier than the unit suite). The margin is not slack to spend, it absorbs the two things that legitimately move a score without anyone editing a test: a Stryker/vitest upgrade that changes which mutants are generated, and timeouts, which count as killed and can flip under CI load. Raise the floor when the score rises; never lower it to make a red run green — a drop means a test stopped discriminating. Measured 81.9% when the floor was set to 80.", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -22,7 +22,7 @@ "thresholds": { "high": 90, "low": 70, - "break": null + "break": 80 }, "concurrency": 2, "timeoutMS": 60000, diff --git a/app/api/stryker.auth.config.json b/app/api/stryker.auth.config.json index b4b06e726..8a7750df6 100644 --- a/app/api/stryker.auth.config.json +++ b/app/api/stryker.auth.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "Mutation testing for the API's credential path. Scoped deliberately to src/auth/ and its OWN unit tests: those are fully mocked, so they run clean inside Stryker's sandbox (which copies app/api only \u2014 tests that read the real filesystem or the tools/crawlers manifests outside it fail the dry run and abort the whole mutation run before any mutant is evaluated). Excluding tests one at a time would risk a false survivor whose killer was skipped; a tight scope avoids that. This is also the fairest possible test of the coverage figure: readTokens.js sits at 100% line AND 100% branch, so its score here says whether that number was earned. Measurement, not a gate \u2014 `break` is null. Run: cd app/api && npx stryker run stryker.auth.config.json MUTATOR SET: StringLiteral and ObjectLiteral are disabled. permissions.js is largely a catalog of labels and descriptions; mutating that text produced 68 of its 72 survivors and dragged the score to 32.7% while saying nothing about behaviour \u2014 no sensible test asserts every description string. Disabling them makes the number mean 'did a wrong DECISION get caught', which is the question worth gating on. The cost is real and worth knowing: SQL strings stop being mutated too, so this cannot tell you a query is correct. That stays the contract tests' job (see app/api/CLAUDE.md \u2014 the unit mocks are SQL-blind by design).", + "_comment": "Mutation testing for the API's credential path. Scoped deliberately to src/auth/ and its OWN unit tests: those are fully mocked, so they run clean inside Stryker's sandbox (which copies app/api only — tests that read the real filesystem or the tools/crawlers manifests outside it fail the dry run and abort the whole mutation run before any mutant is evaluated). Excluding tests one at a time would risk a false survivor whose killer was skipped; a tight scope avoids that. This is also the fairest possible test of the coverage figure: readTokens.js sits at 100% line AND 100% branch, so its score here says whether that number was earned. Enforced: see MUTATION FLOOR below. Run: cd app/api && npm run test:mutation:auth MUTATOR SET: StringLiteral and ObjectLiteral are disabled. permissions.js is largely a catalog of labels and descriptions; mutating that text produced 68 of its 72 survivors and dragged the score to 32.7% while saying nothing about behaviour — no sensible test asserts every description string. Disabling them makes the number mean 'did a wrong DECISION get caught', which is the question worth gating on. The cost is real and worth knowing: SQL strings stop being mutated too, so this cannot tell you a query is correct. That stays the contract tests' job (see app/api/CLAUDE.md — the unit mocks are SQL-blind by design). MUTATION FLOOR: `break` is set a few points under the measured score, and is a RATCHET — it only ever goes up. It is enforced by the weekly .github/workflows/js-mutation.yml (never on a PR: mutation is far heavier than the unit suite). The margin is not slack to spend, it absorbs the two things that legitimately move a score without anyone editing a test: a Stryker/vitest upgrade that changes which mutants are generated, and timeouts, which count as killed and can flip under CI load. Raise the floor when the score rises; never lower it to make a red run green — a drop means a test stopped discriminating. Measured 100.0% when the floor was set to 98.", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -20,7 +20,7 @@ "thresholds": { "high": 90, "low": 70, - "break": null + "break": 98 }, "concurrency": 4, "timeoutMS": 60000, diff --git a/app/api/stryker.effectiveaccess.config.json b/app/api/stryker.effectiveaccess.config.json index 98d9b4fd8..585052e71 100644 --- a/app/api/stryker.effectiveaccess.config.json +++ b/app/api/stryker.effectiveaccess.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "Mutation testing for the effective-access layer: the code that decides WHICH access a principal actually has once inheritance, containment and policy filtering are applied. Scoped the same way as stryker.auth.config.json -- these three files plus their OWN unit tests, which are fully self-contained (no filesystem, no crawler manifests), so they run clean inside Stryker's sandbox. WHY THESE FILES: they sit at 93-99% line coverage and answer a security question, which is the exact combination that made readTokens.js worth measuring -- it was at 100% line AND branch coverage and its mutation score still exposed real defects. A wrong comparison here does not crash anything; it quietly shows a principal access they do not have, or hides access they do. Coverage cannot tell those apart from correct code. MUTATOR SET: StringLiteral and ObjectLiteral disabled, matching the auth config. The cost is the same and worth restating: SQL strings stop being mutated too, so this cannot tell you a query is correct -- that stays the contract tests' job. Measurement, not a gate: `break` is null. Run: cd app/api && npx stryker run stryker.effectiveaccess.config.json", + "_comment": "Mutation testing for the effective-access layer: the code that decides WHICH access a principal actually has once inheritance, containment and policy filtering are applied. Scoped the same way as stryker.auth.config.json -- these three files plus their OWN unit tests, which are fully self-contained (no filesystem, no crawler manifests), so they run clean inside Stryker's sandbox. WHY THESE FILES: they sit at 93-99% line coverage and answer a security question, which is the exact combination that made readTokens.js worth measuring -- it was at 100% line AND branch coverage and its mutation score still exposed real defects. A wrong comparison here does not crash anything; it quietly shows a principal access they do not have, or hides access they do. Coverage cannot tell those apart from correct code. MUTATOR SET: StringLiteral and ObjectLiteral disabled, matching the auth config. The cost is the same and worth restating: SQL strings stop being mutated too, so this cannot tell you a query is correct -- that stays the contract tests' job. Enforced: see MUTATION FLOOR below. Run: cd app/api && npm run test:mutation:effectiveaccess MUTATION FLOOR: `break` is set a few points under the measured score, and is a RATCHET — it only ever goes up. It is enforced by the weekly .github/workflows/js-mutation.yml (never on a PR: mutation is far heavier than the unit suite). The margin is not slack to spend, it absorbs the two things that legitimately move a score without anyone editing a test: a Stryker/vitest upgrade that changes which mutants are generated, and timeouts, which count as killed and can flip under CI load. Raise the floor when the score rises; never lower it to make a red run green — a drop means a test stopped discriminating. Measured 94.7% when the floor was set to 92.", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -22,7 +22,7 @@ "thresholds": { "high": 90, "low": 70, - "break": null + "break": 92 }, "concurrency": 2, "timeoutMS": 60000, diff --git a/app/ui/package.json b/app/ui/package.json index c98b324e1..b2b2488f6 100644 --- a/app/ui/package.json +++ b/app/ui/package.json @@ -17,7 +17,8 @@ "test:e2e:sql": "bash scripts/e2e-sql.sh", "test:e2e:headed": "npx playwright test --headed", "test:e2e:ui": "npx playwright test --ui", - "test:mutation": "stryker run stryker.pilot.config.json" + "test:mutation": "npm run test:mutation:pilot", + "test:mutation:pilot": "stryker run stryker.pilot.config.json" }, "dependencies": { "@azure/msal-browser": "^5.18.0", diff --git a/app/ui/stryker.pilot.config.json b/app/ui/stryker.pilot.config.json index 2d56ec572..59a2245d9 100644 --- a/app/ui/stryker.pilot.config.json +++ b/app/ui/stryker.pilot.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "First mutation measurement of the UI. Two files, picked so the run answers two different questions rather than producing one blended number. src/auth/usePermissions.js is the client-side permission gate: seven components import it, NO test file imported it before this pilot, and its 39% line coverage was incidental — whatever happened to execute while other components rendered. src/utils/matrixFilter.js is the opposite case: 99% coverage with its own dedicated test file, so its score says whether that number was earned. Measurement, not a gate — `break` is null. Run: cd app/ui && npm run test:mutation. MUTATOR SET: everything enabled, which is DELIBERATELY different from app/api/stryker.auth.config.json. There, StringLiteral and ObjectLiteral were disabled because permissions.js is largely a catalog of human-readable labels and mutating that prose produced 68 of 72 survivors while saying nothing about behaviour. Here the string literals ARE the behaviour: 'admin.auth' is the permission a hook checks for, and 'rows-as-resources' is a value the normaliser accepts — a wrong one silently grants or drops. Those mutants are worth killing, so they stay in. Do not copy the API's exclusions here without re-reading this. THE 10 REMAINING SURVIVORS ARE ANALYSED, NOT UNEXAMINED, and are left visible rather than declared away (`break` is null, so they cost nothing): (a) 7 of them are the delimiters inside matrixFilter's `canonical()` -- '[' ',' ']' '{' ':' '}'. Both sides of any fingerprint comparison run through the SAME mutated serializer, so killing one needs two structurally different filters that collide once a delimiter is dropped. JSON.stringify escapes every string it emits, so the quote pattern needed to forge a collision cannot be produced by any value a filter can hold. (b) `Array.isArray(obj) -> false` serialises arrays as index-keyed objects; no filter field can hold either an array or a plain object depending on input (`list()` returns [] for a non-array, normalizeSort always returns an array), so the two shapes never meet in the same slot. (c) `typeof f === 'object' -> true` in normalizeMatrixFilter leaves `f ? f : EMPTY_FILTER`: a truthy non-object (string, number, boolean) has none of the sixteen filter fields on it, so every field falls back and the result is EMPTY_FILTER either way. (d) `permissions.size === 0 -> false` in usePermissions drops a redundant fast path -- an empty Set reaches `.some()` and returns false regardless. Re-check (a) and (b) if canonical() ever stops delegating to JSON.stringify.", + "_comment": "First mutation measurement of the UI. Two files, picked so the run answers two different questions rather than producing one blended number. src/auth/usePermissions.js is the client-side permission gate: seven components import it, NO test file imported it before this pilot, and its 39% line coverage was incidental — whatever happened to execute while other components rendered. src/utils/matrixFilter.js is the opposite case: 99% coverage with its own dedicated test file, so its score says whether that number was earned. Enforced: see MUTATION FLOOR below. Run: cd app/ui && npm run test:mutation. MUTATOR SET: everything enabled, which is DELIBERATELY different from app/api/stryker.auth.config.json. There, StringLiteral and ObjectLiteral were disabled because permissions.js is largely a catalog of human-readable labels and mutating that prose produced 68 of 72 survivors while saying nothing about behaviour. Here the string literals ARE the behaviour: 'admin.auth' is the permission a hook checks for, and 'rows-as-resources' is a value the normaliser accepts — a wrong one silently grants or drops. Those mutants are worth killing, so they stay in. Do not copy the API's exclusions here without re-reading this. THE 10 REMAINING SURVIVORS ARE ANALYSED, NOT UNEXAMINED, and are left visible rather than declared away (the floor sits below the measured score, so they cost nothing): (a) 7 of them are the delimiters inside matrixFilter's `canonical()` -- '[' ',' ']' '{' ':' '}'. Both sides of any fingerprint comparison run through the SAME mutated serializer, so killing one needs two structurally different filters that collide once a delimiter is dropped. JSON.stringify escapes every string it emits, so the quote pattern needed to forge a collision cannot be produced by any value a filter can hold. (b) `Array.isArray(obj) -> false` serialises arrays as index-keyed objects; no filter field can hold either an array or a plain object depending on input (`list()` returns [] for a non-array, normalizeSort always returns an array), so the two shapes never meet in the same slot. (c) `typeof f === 'object' -> true` in normalizeMatrixFilter leaves `f ? f : EMPTY_FILTER`: a truthy non-object (string, number, boolean) has none of the sixteen filter fields on it, so every field falls back and the result is EMPTY_FILTER either way. (d) `permissions.size === 0 -> false` in usePermissions drops a redundant fast path -- an empty Set reaches `.some()` and returns false regardless. Re-check (a) and (b) if canonical() ever stops delegating to JSON.stringify. MUTATION FLOOR: `break` is set a few points under the measured score, and is a RATCHET — it only ever goes up. It is enforced by the weekly .github/workflows/js-mutation.yml (never on a PR: mutation is far heavier than the unit suite). The margin is not slack to spend, it absorbs the two things that legitimately move a score without anyone editing a test: a Stryker/vitest upgrade that changes which mutants are generated, and timeouts, which count as killed and can flip under CI load. Raise the floor when the score rises; never lower it to make a red run green — a drop means a test stopped discriminating. Measured 93.8% when the floor was set to 91.", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -20,7 +20,7 @@ "thresholds": { "high": 90, "low": 70, - "break": null + "break": 91 }, "concurrency": 4, "timeoutMS": 60000, diff --git a/changes/phases-into-gate.md b/changes/phases-into-gate.md index 0b7f7ba3d..2bde64166 100644 --- a/changes/phases-into-gate.md +++ b/changes/phases-into-gate.md @@ -6,3 +6,6 @@ - Continued raising fault detection on the three crawler phase layers toward the gate's threshold, and recorded the progress-bar arithmetic as accepted rather than pretending it is untested — with two entries pulled back out after review, because they turned out to be the system identifier records are attributed to rather than display counters. - Closed further gaps found this way: a tenant, an Omada system and a midPoint resource are each now proven to be registered as enabled and syncable (registered otherwise, a freshly connected system is silently inert and never crawled again); a tenant-wide application consent can no longer be recorded as though an individual user had personally granted it; sign-in activity is uploaded for a tenant where only one person has ever signed in; and identity correlation no longer runs against a filter that names no attribute to correlate on. - Brought account linking — the code that decides which accounts belong to the same person — under fault testing. Both failure directions are silent: link too eagerly and one person inherits another's access everywhere it is shown, link too shyly and their real combined access never appears to a reviewer. Fixed gaps in how email addresses and names are normalised before comparison (an address that was merely uppercase or padded with spaces could fail to match itself), in the ordering that decides which classification rule wins, and in how guest accounts are recognised and explained. +- Fault detection now runs automatically every week for the web interface and API, not just by hand. Four areas are checked — the credential and permission path, the effective-access engine, account correlation, and the two web-interface modules already measured — and each carries a minimum score the build fails below. Previously nothing re-ran these after the person who wrote them moved on, so a test that quietly stopped catching bugs would have gone unnoticed indefinitely. +- The Test Coverage page now shows a fault-detection figure for the web interface and API alongside the one it already showed for the crawlers, and states plainly how much of each area that figure covers — ten files out of roughly four hundred. A high score over a small slice reads very differently once the slice is named, and it was previously not named. +- Corrected several pieces of project documentation that had stopped being true: the recorded threshold for crawler fault detection, the claim that no provably-harmless findings had been recorded (there are 127, each with a written reason), and a status section still describing the crawler phase layers as unmeasured after they had been measured. diff --git a/docs/contributing/writing-tests-that-assert.md b/docs/contributing/writing-tests-that-assert.md index 7777d4908..2a98e3902 100644 --- a/docs/contributing/writing-tests-that-assert.md +++ b/docs/contributing/writing-tests-that-assert.md @@ -172,21 +172,46 @@ an existing test can revive a mutant it never evaluates. **A mutant that provably cannot change behaviour can be declared**, with a reason, in the config's `equivalents` map. The declaration is checked, not merely recorded: the run fails if a declared mutant is ever killed, or if it stops -existing. Nothing in this repo is declared yet — the remaining survivors here are -progress percentages and log constants, and leaving them visible is honest. +existing. There are 127 declarations today, and the reason on each says which +kind it is — most are provable (a hashtable used as a set, a `-Depth` larger than +the payload nests, a `Select-Object -Last 1` over a list that can hold at most one +row), a minority are judgements that a visible-but-arbitrary constant is not worth +a test (a progress counter's starting value, a column width in a format string). +Write which one you are claiming. Two declarations were withdrawn on review after +turning out to be system identifiers rather than display counters, and the giveaway +was that their reasons asserted *provable* for something only argued. ### JavaScript — Stryker -Config: `app/api/stryker.auth.config.json`. Run it via the npm script: +Four scopes today, each its own config plus a narrow `vitest.stryker.*.config.js`: + +| Config | Covers | Score | +|---|---|---| +| `app/api/stryker.auth.config.json` | credential + permission path | 100% | +| `app/api/stryker.effectiveaccess.config.json` | the effective-access engine, its policies and LRU | 94.7% | +| `app/api/stryker.accountlinking.config.json` | account correlation and its rules | 81.9% | +| `app/ui/stryker.pilot.config.json` | `usePermissions`, `matrixFilter` | 93.8% | + +Run one, or all of a package's: ```bash -cd app/api && npm run test:mutation +cd app/api && npm run test:mutation:accountlinking # one scope +cd app/api && npm run test:mutation # all three, in sequence ``` Use the script, not `npx stryker` — npx can resolve the **deprecated standalone `stryker` package** from its cache instead of the installed `@stryker-mutator/core`, failing with `Cannot find module 'rx'`. +Every config carries a `thresholds.break` a few points under its measured score, +enforced weekly by `.github/workflows/js-mutation.yml` (Monday 05:00 UTC, and on +demand — never on a PR, where a run this heavy could not be a required check +anyway). The floors are a ratchet: raise them as scores rise, and never lower one +to make a red run green, because a drop means a test stopped discriminating. That +workflow also publishes the merged per-suite score to the coverage docs page, +carrying its scope note with it — ten files of ~410 eligible, which is why the +number must never be read as suite-wide. + Two constraints worth knowing before you widen the scope: - **Stryker sandboxes `app/api` alone.** Any test reading the real filesystem — @@ -298,25 +323,35 @@ pattern is the same each time: an assertion built on a guess about *shape* or | `test/unit/PSMutationScope.Tests.ps1` | Every eligible crawler file mutated or excluded with a reason; every mutated file names a suite that exercises it | | `test/lib/HttpErrorFixtures.psm1` | One owner for HTTP-error shapes | | `tools/generate-coverage-doc.py` | Generates the scope and shape caveats on the coverage page | -| `.ci/psmutant.config.json` / `app/api/stryker.auth.config.json` | The mutation scope declarations | +| `app/api/src/mutationScope.guard.test.js` | Every eligible JS file mutated, backlogged, or excluded with a reason; the backlog free of stale and already-covered entries | +| `.ci/psmutant.config.json` / `app/*/stryker.*.config.json` | The mutation scope declarations, and the enforced score floors | +| `.github/workflows/ps-mutation.yml` / `js-mutation.yml` | Run both tools weekly against those floors, and publish the scores to the coverage page | --- ## Where the work stands -**In mutation scope:** the crawler shapers, the shared ingest library, the -`*.Functions.ps1` layer, the OData library, the SDK's Graph pager, the -Azure/midPoint helper clients (23 PowerShell files), and the API credential path -(100%). +This section goes stale fast — check the numbers against the coverage page +(`docs/reference/coverage.md`, refreshed by the weekly runs) before quoting them. + +**PowerShell — 112 files in `mutate`, 20 excluded with reasons, floor at 85%.** +The crawler Phases layer used to be the open item here, described as monolithic +and untestable long after it had been decomposed; it is now in scope with the +rest. Watch for that failure mode when you edit this section: the description +outlived the thing it described, and was then cited as a reason not to measure. + +**JavaScript — 10 files of ~410 eligible.** That is the honest headline: the +scores above are high *and* they describe 5% of the API's coverable lines and 1% +of the UI's. The rest is a counted backlog in +`.ci/js-mutation-scope-baseline.json`, guarded so new code cannot join it +silently. A file leaves the backlog by entering a Stryker config's `mutate`, not +by being deleted from the list. **Open:** -- **The crawler Phases layer** — five files at **50.1%**, 337 surviving mutants - across 158 functions. No single cause; it is sustained test-writing. The - technique that moves it is asserting payload content rather than collection - counts. +- **The JS backlog itself** — ~398 files with no fault-detection evidence. Line + coverage is not a proxy: measured pairs here include 97% line / 68% mutation, + 93% / 69%, and 99% / 85%, and `usePermissions.js` sat at 39% line with no test + file importing it at all. - **`tools/powershell-sdk/` and `tools/riskscoring/`** have no eligibility - definition, so the scope guard cannot see them. -- **The UI suite** has no mutation evidence, and its method coverage (69.2%) sits - below its line coverage (80.9%) — the signature of components rendered but - never driven. + definition, so the PowerShell scope guard cannot see them. diff --git a/test/ci-scripts/test-gate-wiring.sh b/test/ci-scripts/test-gate-wiring.sh index 5c6e4ade1..2e907c41b 100644 --- a/test/ci-scripts/test-gate-wiring.sh +++ b/test/ci-scripts/test-gate-wiring.sh @@ -117,11 +117,22 @@ for f in "$WF"/*.yml; do done assert "no unlisted workflow runs on pull_request" "" "${unlisted% }" -# ── 3. The heavy weekly job stays off pull requests ───────────────────────── +# ── 3. The heavy weekly jobs stay off pull requests ───────────────────────── # Mutation testing is far heavier than the unit suite. On PRs it could never be required (see the -# header), so it only ever looked like a gate. -assert "ps-mutation does not run on pull_request" "weekly-only" \ - "$(grep -qE '^[[:space:]]+pull_request:' "$WF/ps-mutation.yml" && echo "runs-on-prs" || echo "weekly-only")" +# header), so it only ever looked like a gate. Derived over every *-mutation.yml, not listed by +# name: adding a mutation workflow for a third language inherits the rule instead of quietly +# skipping it. The glob must also match something — a rename would otherwise pass vacuously. +found=0 +onprs="" +for f in "$WF"/*-mutation.yml; do + [ -e "$f" ] || continue + found=$((found + 1)) + grep -qE '^[[:space:]]+pull_request:' "$f" && onprs="${onprs}$(basename "$f") " +done +# Both mutation workflows must be found. Without this, renaming them to something the glob misses +# would make the check below pass over an empty list — green, and testing nothing. +assert "both mutation workflows are matched by the glob" "2" "$found" +assert "no mutation workflow runs on pull_request" "" "${onprs% }" # ── 4. Anything that commits back to main must survive losing the race ────── # Several workflows push to main off the SAME merge — bump-version and coverage both do, seconds diff --git a/tools/generate-coverage-doc.py b/tools/generate-coverage-doc.py index 0c3d5fa7d..78dd5c8a5 100644 --- a/tools/generate-coverage-doc.py +++ b/tools/generate-coverage-doc.py @@ -20,9 +20,15 @@ the Node suites emit it via ``tools/complexity/ratchet.py --emit-complexity-json`` (ESLint's ``complexity`` rule + eslint-plugin-sonarjs). We aggregate it to avg / max cyclomatic and cognitive complexity per suite. -* **Mutation** — ``//mutation.json``, the PSMutant report - (``{mutationScore, killed, total, …}``): the share of injected faults the tests - actually catch. PowerShell-only today; suites without the file show —. +* **Mutation** — ``//mutation.json``, in the shape PSMutant + emits (``{mutationScore, killed, total, …}``): the share of injected faults the + tests actually catch. PowerShell publishes it straight from PSMutant via the + weekly ps-mutation.yml; the API and UI suites publish it from Stryker via the + weekly js-mutation.yml, which merges each package's per-scope Stryker reports + into that same shape (tools/mutation/stryker_to_mutation_json.py). Both are + weekly, so a suite's mutation figure can be up to a week older than its line + coverage — the scope note says so when the two disagree. Suites without the + file show —. Usage: python3 tools/generate-coverage-doc.py \ @@ -37,6 +43,7 @@ """ import argparse import datetime +import glob import json import os import sys @@ -49,6 +56,17 @@ ("powershell", "PowerShell (Pester)"), ] +# Package root whose stryker*.config.json files DECLARE a JS suite's mutation +# scope. PowerShell declares its scope in one PSMutant config (--psmutant-config); +# the JS suites spread theirs over several Stryker configs in the package root, so +# the declaration is the union of their `mutate` lists. Both are read from the +# committed config rather than from the report for the same reason: the config is +# current as of the merge, the report only as of the last weekly mutation run. +STRYKER_PACKAGES = { + "api": "app/api", + "ui": "app/ui", +} + def load_summary(coverage_dir, slug): """Return the ReportGenerator 'summary' dict for a suite, or None if absent.""" @@ -182,6 +200,48 @@ def declared_mutation_files(config_path): return {"files": set(declared), "operators": len(cfg.get("operators") or []) or None} +def declared_stryker_files(package_dir): + """The scope a JS package DECLARES it mutates: the union of the `mutate` lists + in every stryker*.config.json in its root. + + Same derivation as app/api/src/mutationScope.guard.test.js — discovering the + configs by glob rather than naming them means adding a fourth Stryker scope + widens the declared scope here with no edit, which is what stops this page + understating itself the way it once did for PowerShell (see + declared_mutation_files). Paths in `mutate` are package-relative, matching the + coverage keys ('src/…'), so they need no rewriting. Operators are None: Stryker + declares which mutators to EXCLUDE, not a fixed operator list, so there is no + honest count to report and scope_note simply omits the clause. + + Returns None when the directory holds no readable config with a `mutate` list, + so the caller falls back to what the report actually ran.""" + if not package_dir or not os.path.isdir(package_dir): + return None + files = set() + for path in sorted(glob.glob(os.path.join(package_dir, "stryker*.config.json"))): + try: + with open(path, "r", encoding="utf-8") as fh: + cfg = json.load(fh) + except (OSError, ValueError): + continue + files.update(cfg.get("mutate") or []) + if not files: + return None + return {"files": files, "operators": None} + + +def declared_scope_for(slug, psmutant_config, repo_root="."): + """The committed mutation-scope declaration for one suite, or None. + + PowerShell's lives in a single PSMutant config; the JS suites' in their + package's Stryker configs. Suites with neither (a suite that runs no mutation + testing) get None and fall back to the report.""" + if slug == "powershell": + return declared_mutation_files(psmutant_config) + package = STRYKER_PACKAGES.get(slug) + return declared_stryker_files(os.path.join(repo_root, package)) if package else None + + def resolve_declared_scope(report, declared, measured): """Pick the authoritative file set and operator count for the scope note. @@ -311,14 +371,18 @@ def fmt_avg_max(avg, mx): return f"{avg:.1f} / {mx}" -def collect_rows(coverage_dir, psmutant_config=None): +def collect_rows(coverage_dir, psmutant_config=None, repo_root="."): """Load each suite's ReportGenerator summary plus its optional complexity / mutation side-inputs into ``(slug, label, data|None)`` rows. Returns the rows - along with the running overall covered / coverable line totals.""" + along with the running overall covered / coverable line totals. + + The mutation-scope declaration is read PER SUITE (see declared_scope_for), not + once for the whole page: PowerShell declares its scope in a PSMutant config and + the JS suites in their Stryker configs. Applying one suite's declaration to + another would report every row as measuring the wrong file set.""" rows = [] overall_covered = 0 overall_coverable = 0 - declared_scope = declared_mutation_files(psmutant_config) for slug, label in SUITES: summary = load_summary(coverage_dir, slug) if summary is None: @@ -346,7 +410,8 @@ def collect_rows(coverage_dir, psmutant_config=None): } # Scope + shape diagnostics: what each number is measured over, and where # the aggregate is least representative. See render_diagnostics. - data["mutation_scope"] = mutation_scope(report, files, declared_scope) + data["mutation_scope"] = mutation_scope( + report, files, declared_scope_for(slug, psmutant_config, repo_root)) data["flags"] = shape_flags(data) data["hotspots"] = complexity_hotspots(units, files, branch) rows.append((slug, label, data)) @@ -526,11 +591,15 @@ def main(): help="Committed mutation-scope declaration. Scope is read from here " "(current as of the merge) rather than from the report, which the " "weekly ps-mutation.yml regenerates on its own cadence.") + ap.add_argument("--repo-root", default=".", + help="Repo root, used to find each JS package's stryker*.config.json " + "(the committed declaration of that suite's mutation scope).") ap.add_argument("--report-base", default="../coverage", help="Relative path from the page to the coverage report root.") args = ap.parse_args() - rows, _covered, overall_coverable = collect_rows(args.coverage_dir, args.psmutant_config) + rows, _covered, overall_coverable = collect_rows( + args.coverage_dir, args.psmutant_config, args.repo_root) generated = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M UTC") text = render_markdown(rows, args.report_base, args.commit, generated) diff --git a/tools/mutation/stryker_to_mutation_json.py b/tools/mutation/stryker_to_mutation_json.py new file mode 100644 index 000000000..7355d0772 --- /dev/null +++ b/tools/mutation/stryker_to_mutation_json.py @@ -0,0 +1,176 @@ +#!/usr/bin/env python3 +"""Merge a package's Stryker reports into the mutation.json the coverage docs read. + +`tools/generate-coverage-doc.py` surfaces a Mutation column per suite from +``docs/coverage//mutation.json``, in the shape PSMutant emits +(``{mutationScore, killed, total, mutants:[{File, Status, ...}]}``). PowerShell has +had that column since ps-mutation.yml started publishing; the API and UI rows read +"-" because Stryker's JSON reporter emits a different shape (a per-file map of +mutant arrays, one report per Stryker config). This script is the adapter, run by +the weekly .github/workflows/js-mutation.yml after the four scopes finish. + +WHY IT MERGES RATHER THAN PUBLISHING PER CONFIG: the docs page has one row per +suite, and app/api holds three Stryker configs (auth, effective access, account +linking). Publishing them separately would need three new rows describing arbitrary +slices of one suite; merging gives the suite the same single honest number the +PowerShell row has, and the per-scope detail is kept in ``scopes`` for anyone +reading the file. + +WHICH REPORTS ARE REQUIRED IS DERIVED, NOT LISTED. Every ``stryker*.config.json`` +in the package root names its own output in ``jsonReporter.fileName``, so the set +of reports this script demands comes from the configs themselves - add a fifth +config and the next run requires its report without anyone editing this file. A +missing report is a hard error rather than a quietly smaller merge: a merge that +silently drops a scope publishes a number measured over less code than the page +claims, which is the exact failure the scope note exists to prevent. + +SCORE ARITHMETIC matches Stryker's own ``mutationScore``, so the published figure +agrees with the per-config ``break`` thresholds the workflow enforces: Killed and +Timeout count as detected, Survived and NoCoverage count as missed, and Ignored / +CompileError / RuntimeError are excluded from both - a mutant the runner never got +a verdict on is not evidence either way. NoCoverage counts as MISSED rather than +excluded: a mutant no test reaches is a fault nothing would catch, which is the +thing being measured. + +Every generated mutant is emitted, including the ignored ones. They cost roughly +half the file size and buy correctness at the edge: generate-coverage-doc.py derives +the measured file set from the mutants' own ``File`` entries, so a file whose mutants +were all ignored (a string catalog under `excludedMutations`, say) would vanish from +that set and be reported as scope the score does not cover. + +Usage: + python3 tools/mutation/stryker_to_mutation_json.py \ + --package-dir app/api --out docs/coverage/api/mutation.json +""" +import argparse +import glob +import json +import os +import sys + +# Stryker mutant status -> whether the tests detected the fault. Absent from both +# sets (Ignored, CompileError, RuntimeError) means "no verdict" and is excluded +# from the score entirely. Keep in step with Stryker's own mutationScore, which is +# what the per-config `break` thresholds are compared against. +DETECTED = {"Killed", "Timeout"} +MISSED = {"Survived", "NoCoverage"} + + +def stryker_configs(package_dir): + """Every stryker*.config.json in a package root, sorted for stable output. + + Same discovery rule as app/api/src/mutationScope.guard.test.js, which derives + the mutation-covered file set this way - one convention, not two lists.""" + return sorted(glob.glob(os.path.join(package_dir, "stryker*.config.json"))) + + +def report_path(package_dir, config_path): + """Where a Stryker config writes its JSON report, read from the config itself.""" + with open(config_path, "r", encoding="utf-8") as fh: + cfg = json.load(fh) + name = (cfg.get("jsonReporter") or {}).get("fileName") + if not name: + raise SystemExit( + f"{config_path} has no jsonReporter.fileName - the merge cannot know " + "where its report lands. Add one, or drop the config." + ) + return os.path.join(package_dir, name) + + +def read_mutants(path): + """Flatten a Stryker report to ``[{File, Line, Status, Operator}, ...]``.""" + with open(path, "r", encoding="utf-8") as fh: + report = json.load(fh) + out = [] + for name, entry in sorted((report.get("files") or {}).items()): + for mutant in entry.get("mutants") or []: + start = (mutant.get("location") or {}).get("start") or {} + out.append({ + "File": name, + "Line": start.get("line"), + "Status": mutant.get("status"), + "Operator": mutant.get("mutatorName"), + }) + return out + + +def tally(mutants): + """(killed, survived) over a mutant list, using Stryker's own score rule.""" + killed = sum(1 for m in mutants if m["Status"] in DETECTED) + survived = sum(1 for m in mutants if m["Status"] in MISSED) + return killed, survived + + +def score(killed, survived): + """Percentage detected, or None when nothing was measured - so an empty run + publishes no number rather than a 0% or a 100% that both read as a verdict.""" + total = killed + survived + return None if total == 0 else round(100.0 * killed / total, 2) + + +def merge(package_dir): + """Merge every configured scope in a package into one PSMutant-shaped report.""" + configs = stryker_configs(package_dir) + if not configs: + raise SystemExit(f"No stryker*.config.json under {package_dir} - nothing to merge.") + + mutants, scopes, missing = [], [], [] + for config_path in configs: + path = report_path(package_dir, config_path) + if not os.path.isfile(path): + missing.append(f"{os.path.basename(config_path)} -> {path}") + continue + found = read_mutants(path) + killed, survived = tally(found) + mutants.extend(found) + scopes.append({ + "config": os.path.basename(config_path), + "score": score(killed, survived), + "killed": killed, + "survived": survived, + }) + + if missing: + raise SystemExit( + "Missing Stryker report(s) - the merge would publish a score measured " + "over less code than the docs page claims:\n " + "\n ".join(missing) + ) + + killed, survived = tally(mutants) + return { + "tool": "stryker", + "mutationScore": score(killed, survived), + "killed": killed, + "survived": survived, + "total": killed + survived, + "scopes": scopes, + "mutants": mutants, + } + + +def main(argv=None): + ap = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + ap.add_argument("--package-dir", required=True, + help="Package root holding the stryker*.config.json files (e.g. app/api).") + ap.add_argument("--out", required=True, + help="Where to write the merged report (e.g. docs/coverage/api/mutation.json).") + args = ap.parse_args(argv) + + merged = merge(args.package_dir) + os.makedirs(os.path.dirname(args.out) or ".", exist_ok=True) + with open(args.out, "w", encoding="utf-8", newline="\n") as fh: + json.dump(merged, fh, indent=2, ensure_ascii=False) + fh.write("\n") + + print(f"[mutation] {args.package_dir}: {merged['mutationScore']}% " + f"({merged['killed']} killed / {merged['total']}) across " + f"{len(merged['scopes'])} scope(s) -> {args.out}") + for entry in merged["scopes"]: + detected = entry["killed"] + entry["survived"] + print(f" {entry['config']}: {entry['score']}% " + f"({entry['killed']} killed / {detected})") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/mutation/test_stryker_to_mutation_json.py b/tools/mutation/test_stryker_to_mutation_json.py new file mode 100644 index 000000000..399142bdc --- /dev/null +++ b/tools/mutation/test_stryker_to_mutation_json.py @@ -0,0 +1,217 @@ +"""Unit tests for tools/mutation/stryker_to_mutation_json.py. + +The script publishes a number to the coverage docs page, so the tests here are +chosen to discriminate rather than to execute: every status fixture uses a +DIFFERENT count per bucket, so a mapping that swaps two statuses (or drops one +from the denominator) produces a different score and fails, instead of landing on +the same arithmetic by luck. + +Run: python -m pytest tools/mutation/test_stryker_to_mutation_json.py -q +""" +import importlib.util +import json +import os + +import pytest + +_HERE = os.path.dirname(os.path.abspath(__file__)) +_SPEC = importlib.util.spec_from_file_location( + "stryker_to_mutation_json", os.path.join(_HERE, "stryker_to_mutation_json.py") +) +conv = importlib.util.module_from_spec(_SPEC) +_SPEC.loader.exec_module(conv) + + +# ── fixtures ───────────────────────────────────────────────────────────────── + +def mutant(status, line=1, mutator="ConditionalExpression"): + """A mutant as STRYKER writes it (lowercase keys) — report-file input.""" + return {"status": status, "location": {"start": {"line": line}}, "mutatorName": mutator} + + +def flat(status): + """A mutant as read_mutants HANDS ON (capitalised keys) — tally/score input. + + Two shapes, two helpers, on purpose: passing a raw Stryker mutant to tally() + is a bug the tests should not be able to hide behind a permissive lookup.""" + return {"File": "src/a.js", "Line": 1, "Status": status, "Operator": "X"} + + +def write_report(path, files): + """files: {name: [mutant, ...]} -> a minimal Stryker JSON report.""" + os.makedirs(os.path.dirname(path), exist_ok=True) + with open(path, "w", encoding="utf-8") as fh: + json.dump({"files": {n: {"mutants": m} for n, m in files.items()}}, fh) + + +def write_config(package_dir, name, mutate, report_name): + os.makedirs(package_dir, exist_ok=True) + path = os.path.join(package_dir, name) + with open(path, "w", encoding="utf-8") as fh: + json.dump({"mutate": mutate, "jsonReporter": {"fileName": report_name}}, fh) + return path + + +# ── tally: which statuses count, and on which side ─────────────────────────── + +def test_tally_counts_timeout_as_detected_and_nocoverage_as_missed(): + # Deliberately unequal buckets: 3 Killed + 1 Timeout = 4 detected, + # 2 Survived + 1 NoCoverage = 3 missed. Every plausible mis-mapping + # (Timeout missed, NoCoverage excluded, NoCoverage detected) yields a + # different pair, so this fixture can tell them apart. + mutants = ([flat("Killed")] * 3 + [flat("Timeout")] + + [flat("Survived")] * 2 + [flat("NoCoverage")]) + assert conv.tally(mutants) == (4, 3) + + +def test_tally_excludes_verdictless_statuses_from_both_sides(): + # Ignored/CompileError/RuntimeError are not evidence either way. Five of them + # against one Killed and two Survived: if any leaked into a bucket the pair + # would move, and the count differs per status so it says WHICH leaked. + mutants = ([flat("Killed")] + [flat("Survived")] * 2 + + [flat("Ignored")] * 5 + [flat("CompileError")] * 3 + + [flat("RuntimeError")] * 4) + assert conv.tally(mutants) == (1, 2) + + +def test_tally_of_nothing_is_zero_zero(): + assert conv.tally([]) == (0, 0) + + +# ── score ──────────────────────────────────────────────────────────────────── + +def test_score_is_the_detected_share_rounded_to_two_places(): + assert conv.score(1, 2) == 33.33 # 33.333... -> two places + assert conv.score(2, 1) == 66.67 # and rounds up, not truncates + assert conv.score(151, 10) == 93.79 + + +def test_score_is_none_when_nothing_was_measured(): + # Not 0.0 and not 100.0 — both read as a verdict on the page. None makes the + # Mutation cell fall back to "—", which is the true statement. + assert conv.score(0, 0) is None + assert conv.score(0, 1) == 0.0 # a real 0% is still a real number + assert conv.score(1, 0) == 100.0 + + +# ── read_mutants ───────────────────────────────────────────────────────────── + +def test_read_mutants_flattens_every_file_with_line_and_operator(tmp_path): + path = str(tmp_path / "r.json") + write_report(path, { + "src/b.js": [mutant("Killed", line=7, mutator="ArithmeticOperator")], + "src/a.js": [mutant("Survived", line=3, mutator="BooleanLiteral")], + }) + got = conv.read_mutants(path) + # Sorted by file name, so output is stable across runs and diffs stay readable. + assert [m["File"] for m in got] == ["src/a.js", "src/b.js"] + assert got[0] == {"File": "src/a.js", "Line": 3, + "Status": "Survived", "Operator": "BooleanLiteral"} + assert got[1]["Line"] == 7 and got[1]["Operator"] == "ArithmeticOperator" + + +def test_read_mutants_keeps_ignored_mutants_so_the_file_set_stays_complete(tmp_path): + # The docstring's edge case, and the reason ignored mutants are emitted at all: + # generate-coverage-doc.py derives the MEASURED file set from mutants[].File. + # Drop the ignored ones and a file with no verdict-carrying mutant disappears + # from that set, and the page reports scope the score supposedly does not cover. + path = str(tmp_path / "r.json") + write_report(path, { + "src/catalog.js": [mutant("Ignored"), mutant("Ignored")], + "src/real.js": [mutant("Killed")], + }) + got = conv.read_mutants(path) + assert {m["File"] for m in got} == {"src/catalog.js", "src/real.js"} + assert conv.tally(got) == (1, 0) # ...while contributing nothing to the score + + +def test_read_mutants_tolerates_a_report_with_no_files_or_no_location(tmp_path): + path = str(tmp_path / "empty.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump({"files": {"src/a.js": {"mutants": [{"status": "Killed"}]}}}, fh) + assert conv.read_mutants(path) == [ + {"File": "src/a.js", "Line": None, "Status": "Killed", "Operator": None} + ] + + bare = str(tmp_path / "bare.json") + with open(bare, "w", encoding="utf-8") as fh: + json.dump({}, fh) + assert conv.read_mutants(bare) == [] + + +# ── report_path ────────────────────────────────────────────────────────────── + +def test_report_path_reads_the_configs_own_reporter_filename(tmp_path): + pkg = str(tmp_path / "app") + cfg = write_config(pkg, "stryker.a.config.json", ["src/a.js"], "reports/out.json") + assert conv.report_path(pkg, cfg) == os.path.join(pkg, "reports/out.json") + + +def test_report_path_refuses_a_config_that_names_no_report(tmp_path): + pkg = str(tmp_path / "app") + os.makedirs(pkg) + cfg = os.path.join(pkg, "stryker.a.config.json") + with open(cfg, "w", encoding="utf-8") as fh: + json.dump({"mutate": ["src/a.js"]}, fh) + with pytest.raises(SystemExit, match="jsonReporter.fileName"): + conv.report_path(pkg, cfg) + + +# ── merge ──────────────────────────────────────────────────────────────────── + +def test_merge_combines_scopes_and_keeps_each_scopes_own_score(tmp_path): + pkg = str(tmp_path / "app") + write_config(pkg, "stryker.one.config.json", ["src/a.js"], "reports/one.json") + write_config(pkg, "stryker.two.config.json", ["src/b.js"], "reports/two.json") + # Deliberately different per-scope scores (100% and 50%) so a bug that reports + # the blended figure per scope, or one scope's figure for the other, fails. + write_report(os.path.join(pkg, "reports/one.json"), + {"src/a.js": [mutant("Killed"), mutant("Killed"), mutant("Killed")]}) + write_report(os.path.join(pkg, "reports/two.json"), + {"src/b.js": [mutant("Killed"), mutant("Survived")]}) + + merged = conv.merge(pkg) + assert merged["killed"] == 4 and merged["survived"] == 1 and merged["total"] == 5 + assert merged["mutationScore"] == 80.0 # 4/5 blended, not the mean of 100 and 50 + assert [(s["config"], s["score"]) for s in merged["scopes"]] == [ + ("stryker.one.config.json", 100.0), + ("stryker.two.config.json", 50.0), + ] + assert {m["File"] for m in merged["mutants"]} == {"src/a.js", "src/b.js"} + + +def test_merge_refuses_to_publish_when_a_scopes_report_is_absent(tmp_path): + pkg = str(tmp_path / "app") + write_config(pkg, "stryker.one.config.json", ["src/a.js"], "reports/one.json") + write_config(pkg, "stryker.two.config.json", ["src/b.js"], "reports/two.json") + write_report(os.path.join(pkg, "reports/one.json"), {"src/a.js": [mutant("Killed")]}) + # two.json never written — publishing now would report 100% for a package whose + # second scope was never measured. + with pytest.raises(SystemExit, match="stryker.two.config.json"): + conv.merge(pkg) + + +def test_merge_refuses_a_package_with_no_stryker_configs(tmp_path): + pkg = str(tmp_path / "app") + os.makedirs(pkg) + with pytest.raises(SystemExit, match="nothing to merge"): + conv.merge(pkg) + + +# ── main ───────────────────────────────────────────────────────────────────── + +def test_main_writes_a_report_generate_coverage_doc_can_read(tmp_path, capsys): + pkg = str(tmp_path / "app") + write_config(pkg, "stryker.one.config.json", ["src/a.js"], "reports/one.json") + write_report(os.path.join(pkg, "reports/one.json"), + {"src/a.js": [mutant("Killed"), mutant("Timeout"), mutant("NoCoverage")]}) + out = str(tmp_path / "docs" / "api" / "mutation.json") + + assert conv.main(["--package-dir", pkg, "--out", out]) == 0 + + written = json.load(open(out, encoding="utf-8")) + # The four keys generate-coverage-doc.py actually consumes. + assert written["mutationScore"] == 66.67 # 2 detected of 3, NoCoverage counting as missed + assert written["killed"] == 2 and written["total"] == 3 + assert {m["File"] for m in written["mutants"]} == {"src/a.js"} + assert "66.67%" in capsys.readouterr().out diff --git a/tools/test_generate_coverage_doc.py b/tools/test_generate_coverage_doc.py index b78251947..ecb1e38a1 100644 --- a/tools/test_generate_coverage_doc.py +++ b/tools/test_generate_coverage_doc.py @@ -266,6 +266,100 @@ def test_declared_mutation_files_none_when_unusable(tmp_path): assert gcd.declared_mutation_files(str(empty)) is None +# ── declared_stryker_files / declared_scope_for ─────────────────────────────── +# The JS suites declare their mutation scope across SEVERAL Stryker configs in a +# package root, where PowerShell declares its in one PSMutant config. These cover +# the union, and the routing that keeps one suite's declaration out of another's +# row — the page previously applied the PowerShell `mutate` list to every suite. + +def _stryker(dirpath, name, mutate): + _write(os.path.join(dirpath, name), {"mutate": mutate, + "jsonReporter": {"fileName": "reports/r.json"}}) + + +def test_declared_stryker_files_unions_every_config_in_the_package(tmp_path): + pkg = str(tmp_path / "app") + _stryker(pkg, "stryker.auth.config.json", ["src/a.js", "src/shared.js"]) + _stryker(pkg, "stryker.other.config.json", ["src/b.js", "src/shared.js"]) + d = gcd.declared_stryker_files(pkg) + # Union, de-duplicated — three distinct files across two configs, not four. + assert d["files"] == {"src/a.js", "src/b.js", "src/shared.js"} + # Stryker declares which mutators to EXCLUDE, not a fixed operator list, so + # there is no honest count and scope_note must omit the clause. + assert d["operators"] is None + + +def test_declared_stryker_files_ignores_a_broken_config_but_keeps_the_rest(tmp_path): + pkg = str(tmp_path / "app") + _stryker(pkg, "stryker.good.config.json", ["src/a.js"]) + with open(os.path.join(pkg, "stryker.bad.config.json"), "w", encoding="utf-8") as fh: + fh.write("{ not json") + # One unparseable config must not blank out the whole declaration — that would + # silently fall back to the report and hide the real scope. + assert gcd.declared_stryker_files(pkg)["files"] == {"src/a.js"} + + +def test_declared_stryker_files_none_when_there_is_nothing_to_read(tmp_path): + assert gcd.declared_stryker_files(None) is None + assert gcd.declared_stryker_files(str(tmp_path / "missing")) is None + empty = str(tmp_path / "empty") + os.makedirs(empty) + assert gcd.declared_stryker_files(empty) is None # no configs at all + _stryker(empty, "stryker.a.config.json", []) + assert gcd.declared_stryker_files(empty) is None # configs, empty mutate + # A non-Stryker JSON file in the package root is not a declaration. + _write(os.path.join(empty, "package.json"), {"mutate": ["src/nope.js"]}) + assert gcd.declared_stryker_files(empty) is None + + +def test_declared_scope_for_routes_each_suite_to_its_own_declaration(tmp_path): + root = str(tmp_path) + _stryker(os.path.join(root, "app", "api"), "stryker.a.config.json", ["src/api.js"]) + _stryker(os.path.join(root, "app", "ui"), "stryker.b.config.json", ["src/ui.js"]) + ps_cfg = str(tmp_path / "psmutant.config.json") + _write(ps_cfg, {"mutate": ["crawler.ps1"], "operators": ["X"]}) + + # Each suite gets ITS OWN file list. The bug this guards: the PowerShell + # declaration was read once and handed to every row, so the API and UI rows + # reported PowerShell's 112 files as the scope of a JS score. + assert gcd.declared_scope_for("api", ps_cfg, root)["files"] == {"src/api.js"} + assert gcd.declared_scope_for("ui", ps_cfg, root)["files"] == {"src/ui.js"} + assert gcd.declared_scope_for("powershell", ps_cfg, root)["files"] == {"crawler.ps1"} + # A suite with neither kind of declaration falls back to the report. + assert gcd.declared_scope_for("unknown-suite", ps_cfg, root) is None + + +def test_collect_rows_scopes_a_js_suite_by_its_own_stryker_configs(tmp_path): + root = tmp_path / "repo" + cov = root / "coverage" + _write(str(cov / "api" / "Summary.json"), { + "summary": {"linecoverage": 90.0, "coveredlines": 90, "coverablelines": 100}, + "coverage": {"assemblies": [{"classesinassembly": [ + {"name": "src/a.js", "coverage": 90.0, "coverablelines": 40}, + {"name": "src/b.js", "coverage": 80.0, "coverablelines": 60}, + ]}]}, + }) + # The report measured ONE file; the configs declare TWO. Different numbers on + # purpose: a row that reported 1 file would mean the declaration was ignored. + _write(str(cov / "api" / "mutation.json"), { + "mutationScore": 75.0, "killed": 3, "total": 4, + "mutants": [{"File": "src/a.js"}], + }) + _stryker(str(root / "app" / "api"), "stryker.one.config.json", ["src/a.js"]) + _stryker(str(root / "app" / "api"), "stryker.two.config.json", ["src/b.js"]) + ps_cfg = str(root / "psmutant.config.json") + _write(ps_cfg, {"mutate": ["crawler-1.ps1", "crawler-2.ps1", "crawler-3.ps1"]}) + + rows, _covered, _coverable = gcd.collect_rows(str(cov), ps_cfg, str(root)) + scope = {slug: data for slug, _, data in rows}["api"]["mutation_scope"] + + assert scope["files"] == 2 # the Stryker declaration, not the 3 PS files + assert scope["measured_files"] == 1 # ...and the report is behind it + assert scope["stale"] is True # so the page says the score lags the scope + assert scope["lines"] == 100 # both declared files matched a coverage entry + assert scope["unmatched"] == 0 + + def test_resolve_declared_scope_prefers_the_declaration_then_the_report(): report = {"operators": ["X", "Y"]} declared = {"files": {"a.ps1"}, "operators": 4} From 8dcf3d06381e8fddb68e1017bd46c212fa038420 Mon Sep 17 00:00:00 2001 From: Taeke Date: Wed, 19 Aug 2026 12:14:34 +0200 Subject: [PATCH 05/15] docs: give this PR its own changelog fragment bump-version.yml merges every changes/*.md into CHANGES.md and then DELETES the fragments. A stack sharing one cumulative file therefore republishes the lower PRs' bullets every time a higher one merges: when #1062 lands, changes/phases-into-gate.md is consumed, and this branch would re-add it carrying those same bullets plus its own. Each branch now owns a fragment named after itself, holding only the bullets it added -- which is what "uniquely named fragment file" in the workflow header means, and what makes its merge-conflict-free claim true for a stack rather than only for parallel branches. --- changes/js-mutation-ci.md | 3 +++ changes/phases-into-gate.md | 3 --- 2 files changed, 3 insertions(+), 3 deletions(-) create mode 100644 changes/js-mutation-ci.md diff --git a/changes/js-mutation-ci.md b/changes/js-mutation-ci.md new file mode 100644 index 000000000..a19c63d4f --- /dev/null +++ b/changes/js-mutation-ci.md @@ -0,0 +1,3 @@ +- Fault detection now runs automatically every week for the web interface and API, not just by hand. Four areas are checked — the credential and permission path, the effective-access engine, account correlation, and the two web-interface modules already measured — and each carries a minimum score the build fails below. Previously nothing re-ran these after the person who wrote them moved on, so a test that quietly stopped catching bugs would have gone unnoticed indefinitely. +- The Test Coverage page now shows a fault-detection figure for the web interface and API alongside the one it already showed for the crawlers, and states plainly how much of each area that figure covers — ten files out of roughly four hundred. A high score over a small slice reads very differently once the slice is named, and it was previously not named. +- Corrected several pieces of project documentation that had stopped being true: the recorded threshold for crawler fault detection, the claim that no provably-harmless findings had been recorded (there are 127, each with a written reason), and a status section still describing the crawler phase layers as unmeasured after they had been measured. diff --git a/changes/phases-into-gate.md b/changes/phases-into-gate.md index 2bde64166..0b7f7ba3d 100644 --- a/changes/phases-into-gate.md +++ b/changes/phases-into-gate.md @@ -6,6 +6,3 @@ - Continued raising fault detection on the three crawler phase layers toward the gate's threshold, and recorded the progress-bar arithmetic as accepted rather than pretending it is untested — with two entries pulled back out after review, because they turned out to be the system identifier records are attributed to rather than display counters. - Closed further gaps found this way: a tenant, an Omada system and a midPoint resource are each now proven to be registered as enabled and syncable (registered otherwise, a freshly connected system is silently inert and never crawled again); a tenant-wide application consent can no longer be recorded as though an individual user had personally granted it; sign-in activity is uploaded for a tenant where only one person has ever signed in; and identity correlation no longer runs against a filter that names no attribute to correlate on. - Brought account linking — the code that decides which accounts belong to the same person — under fault testing. Both failure directions are silent: link too eagerly and one person inherits another's access everywhere it is shown, link too shyly and their real combined access never appears to a reviewer. Fixed gaps in how email addresses and names are normalised before comparison (an address that was merely uppercase or padded with spaces could fail to match itself), in the ordering that decides which classification rule wins, and in how guest accounts are recognised and explained. -- Fault detection now runs automatically every week for the web interface and API, not just by hand. Four areas are checked — the credential and permission path, the effective-access engine, account correlation, and the two web-interface modules already measured — and each carries a minimum score the build fails below. Previously nothing re-ran these after the person who wrote them moved on, so a test that quietly stopped catching bugs would have gone unnoticed indefinitely. -- The Test Coverage page now shows a fault-detection figure for the web interface and API alongside the one it already showed for the crawlers, and states plainly how much of each area that figure covers — ten files out of roughly four hundred. A high score over a small slice reads very differently once the slice is named, and it was previously not named. -- Corrected several pieces of project documentation that had stopped being true: the recorded threshold for crawler fault detection, the claim that no provably-harmless findings had been recorded (there are 127, each with a written reason), and a status section still describing the crawler phase layers as unmeasured after they had been measured. From 16878661c0670bd6f127bc4501980c95f7314775 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 15:01:14 +0200 Subject: [PATCH 06/15] test: bring the matrix layer into the fault-detection gate The matrix layer decides which access is shown once inheritance, context rollups and attribute cuts are applied, and inheritedAccess.js decides whether a grant reads as held directly or through a group -- the thing a reviewer acts on. It was picked as the next scope because it matched the profile of the two worst surprises so far (effectiveAccess/engine.js 93% line -> 69% mutation, accountlinking/classifier.js 97% -> 68%). It is worse than both. 1,127 mutants over 8 files: 63.44% detected against a suite line coverage of 94.3%, a 31-point gap and the widest recorded here. file lines branch funcs mutation scopeHistory.js 84.9% 62.0% 85.7% 51.66% filterSql.js 87.8% 73.8% 91.7% 56.52% rollupBuilders.js 100.0% 100.0% 100.0% 61.67% inheritedAccess.js 97.6% 66.1% 92.3% 65.14% contextRollup.js 100.0% 95.5% 100.0% 68.80% attributeCut.js 100.0% 88.0% 100.0% 70.93% resourceContexts.js 100.0% 100.0% 100.0% 85.71% attrExpr.js 100.0% 100.0% 100.0% 93.48% rollupBuilders.js is the sharpest case in the repo: perfect coverage on every axis, and 38% of injected faults go unnoticed. The 103 no-coverage mutants are NOT an artifact of this config's narrow test include, which was the obvious suspicion and would have made the number a lie. Checked rather than assumed: running the FULL API suite with coverage limited to src/matrix reproduces the per-file figures exactly -- same 94.26% lines, same 76.24% branch, same uncovered lines. The include loses nothing, because the route tests that also drive these modules are in it deliberately (an excluded killer surfaces as a false survivor, which is worse than measuring less). Mutators stay fully enabled rather than inheriting the auth config's StringLiteral/ObjectLiteral exclusions. These are SQL builders and the unit mocks are SQL-blind, so the exclusion looked justified -- but the tests here assert on the emitted SQL directly, so a mutated literal dies. Carry the caveat with the number though: pinning SQL text proves it is unchanged, not that the query returns the right rows. That stays the contract tests' job. Floor set at 61, just under the measurement, and ratchets up from there. --- .ci/js-mutation-scope-baseline.json | 8 ----- .github/workflows/js-mutation.yml | 10 +++--- app/api/package.json | 5 +-- app/api/stryker.matrix.config.json | 34 +++++++++++++++++++ app/api/vitest.stryker.matrix.config.js | 45 +++++++++++++++++++++++++ 5 files changed, 88 insertions(+), 14 deletions(-) create mode 100644 app/api/stryker.matrix.config.json create mode 100644 app/api/vitest.stryker.matrix.config.js diff --git a/.ci/js-mutation-scope-baseline.json b/.ci/js-mutation-scope-baseline.json index d0ad65e22..dd396eddf 100644 --- a/.ci/js-mutation-scope-baseline.json +++ b/.ci/js-mutation-scope-baseline.json @@ -62,14 +62,6 @@ "app/api/src/llm/riskPrompts.js", "app/api/src/llm/scraper.js", "app/api/src/llm/service.js", - "app/api/src/matrix/attrExpr.js", - "app/api/src/matrix/attributeCut.js", - "app/api/src/matrix/contextRollup.js", - "app/api/src/matrix/filterSql.js", - "app/api/src/matrix/inheritedAccess.js", - "app/api/src/matrix/resourceContexts.js", - "app/api/src/matrix/rollupBuilders.js", - "app/api/src/matrix/scopeHistory.js", "app/api/src/middleware/auth.js", "app/api/src/middleware/crawlerAuth.helpers.js", "app/api/src/middleware/crawlerAuth.js", diff --git a/.github/workflows/js-mutation.yml b/.github/workflows/js-mutation.yml index b668a3f6a..b202bc4e2 100644 --- a/.github/workflows/js-mutation.yml +++ b/.github/workflows/js-mutation.yml @@ -19,14 +19,15 @@ # time; that is the deliberate trade for the runtime, same as ps-mutation.yml. # test/ci-scripts/test-gate-wiring.sh enforces this for every *-mutation.yml. # -# SCOPE IS SMALL AND THAT IS THE POINT OF PUBLISHING IT. Ten API/UI files are +# SCOPE IS SMALL AND THAT IS THE POINT OF PUBLISHING IT. Twenty API/UI files are # mutation-tested out of ~410 eligible; the remaining backlog is counted in # .ci/js-mutation-scope-baseline.json and guarded by app/api/src/mutationScope. # guard.test.js, which fails when new code is added without a decision. Publishing -# the score to the coverage docs page carries the scope note with it ("covers 10 -# file(s) of 189 -- 5% of the suite's coverable lines"), so the number can never be +# the score to the coverage docs page carries the scope note with it ("covers N +# file(s) of 189 -- X% of the suite's coverable lines"), so the number can never be # mistaken for a suite-wide one. A quiet 88% would have been the more flattering -# option and the dishonest one. +# option and the dishonest one. Do not hand-edit that sentence's numbers here -- +# generate-coverage-doc.py derives them from the configs on every run. # # Like ps-mutation.yml, this owns the Mutation column for its suites on the Test # Coverage docs page: the publish job merges each package's per-scope reports into @@ -69,6 +70,7 @@ jobs: - { pkg: api, name: auth, label: 'API auth' } - { pkg: api, name: effectiveaccess, label: 'API effective access' } - { pkg: api, name: accountlinking, label: 'API account linking' } + - { pkg: api, name: matrix, label: 'API matrix' } - { pkg: ui, name: pilot, label: 'UI' } steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 diff --git a/app/api/package.json b/app/api/package.json index 03e62e3b8..09dc2e9c9 100644 --- a/app/api/package.json +++ b/app/api/package.json @@ -17,10 +17,11 @@ "postinstall": "patch-package || true", "build:node-launcher": "node ../desktop/scripts/build-node-launcher.mjs", "build:node-launcher:skip-ui": "node ../desktop/scripts/build-node-launcher.mjs --skip-ui-build", - "test:mutation": "npm run test:mutation:auth && npm run test:mutation:effectiveaccess && npm run test:mutation:accountlinking", + "test:mutation": "npm run test:mutation:auth && npm run test:mutation:effectiveaccess && npm run test:mutation:accountlinking && npm run test:mutation:matrix", "test:mutation:auth": "stryker run stryker.auth.config.json", "test:mutation:effectiveaccess": "stryker run stryker.effectiveaccess.config.json", - "test:mutation:accountlinking": "stryker run stryker.accountlinking.config.json" + "test:mutation:accountlinking": "stryker run stryker.accountlinking.config.json", + "test:mutation:matrix": "stryker run stryker.matrix.config.json" }, "devDependencies": { "@eslint/js": "^10.0.1", diff --git a/app/api/stryker.matrix.config.json b/app/api/stryker.matrix.config.json new file mode 100644 index 000000000..843e5b868 --- /dev/null +++ b/app/api/stryker.matrix.config.json @@ -0,0 +1,34 @@ +{ + "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", + "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR starts at 61, just under the first measurement, and only ratchets up -- raise it as survivors are killed.", + "packageManager": "npm", + "testRunner": "vitest", + "vitest": { + "configFile": "vitest.stryker.matrix.config.js" + }, + "reporters": [ + "clear-text", + "json" + ], + "jsonReporter": { + "fileName": "reports/stryker-matrix.json" + }, + "mutate": [ + "src/matrix/attrExpr.js", + "src/matrix/attributeCut.js", + "src/matrix/contextRollup.js", + "src/matrix/filterSql.js", + "src/matrix/inheritedAccess.js", + "src/matrix/resourceContexts.js", + "src/matrix/rollupBuilders.js", + "src/matrix/scopeHistory.js" + ], + "thresholds": { + "high": 90, + "low": 70, + "break": 61 + }, + "concurrency": 4, + "timeoutMS": 60000, + "disableTypeChecks": false +} diff --git a/app/api/vitest.stryker.matrix.config.js b/app/api/vitest.stryker.matrix.config.js new file mode 100644 index 000000000..d5d13af3d --- /dev/null +++ b/app/api/vitest.stryker.matrix.config.js @@ -0,0 +1,45 @@ +import { defineConfig } from 'vitest/config'; +import base from './vitest.config.js'; + +// Vitest config for the matrix mutation run (stryker.matrix.config.json). +// +// Narrow include, for the same reason as the other vitest.stryker.* configs: Stryker +// copies app/api into a temp sandbox, so any test that reads the real filesystem or the +// crawler manifests at ../../tools/crawlers resolves a path that does not exist there, +// fails the dry run, and aborts the whole run before a single mutant is evaluated. +// +// WIDER THAN THE MUTATED DIRECTORY, ON PURPOSE. `src/matrix/**` alone would be the +// obvious include and would be wrong: six of the eight modules are also reached +// transitively through the route layer, and a mutant whose only killer sits in a route +// test would come back a SURVIVOR. That is the failure this narrowing is most likely to +// cause, and a wrong number is worse than a smaller true one — so every direct importer's +// tests are included: +// +// src/routes/matrix/data.js -> attrExpr, inheritedAccess, contextRollup, +// attributeCut, resourceContexts, rollupBuilders +// src/routes/matrix/scope.js -> scopeHistory, attrExpr +// src/routes/matrix/shared.js -> filterSql (and via it src/routes/permissions/) +// src/routes/matrix/savedFilters.js, src/routes/matrix.js -> filterSql, inheritedAccess +// src/routes/resources.js -> resourceContexts +// +// src/db/matrixHelpers.test.js is deliberately NOT here: matrixHelpers.js imports none of +// the eight, so its tests can kill nothing and would only add runtime. +// +// If you add a module to this scope's `mutate` list, trace its importers and widen this +// include to match, rather than assuming the co-located test is the only killer. + +export default defineConfig({ + ...base, + test: { + ...base.test, + include: [ + 'src/matrix/**/*.test.js', + 'src/routes/matrix/**/*.test.js', + 'src/routes/matrix.*.test.js', + 'src/routes/resources.test.js', + 'src/routes/permissions/**/*.test.js', + ], + exclude: ['**/node_modules/**'], + coverage: { ...base.test.coverage, thresholds: undefined }, + }, +}); From f21d0b8f4d5c54fe104ec0dc2f4b35a180e93613 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 15:20:21 +0200 Subject: [PATCH 07/15] test: close the inherited-access gaps the mutation run exposed inheritedAccess.js 65.14% -> 77.64%; the matrix scope 63.44% -> 67.02%, so the floor ratchets 61 -> 65. Each group below was verified by re-applying Stryker's exact reported replacement and confirming the tests fail -- not by assuming a new test covers what it looks like it covers. propagationScope (17 mutants, none previously reached). `reaches()` decides which ancestor actually grants a person's access: per migration 038 an assignment counts at the focus node only when its scope includes `self`, and on an ancestor only when it includes `descendants`. No test varied the scope, so the whole rule could have been inverted silently -- the "why does this person have access" panel naming a resource that grants nothing, or omitting the one that does. The main fixture is five rows that are each the only example of their case, asserted by WHICH ids come back; a count could not tell the right two from a different two. Group principals (9 mutants). A group is how access is delivered, not someone who has it, and three separate sites drop group principals from holder rows and counts. Every existing fixture was a `User`, so all three filters passed everything through and deleting them changed no result -- the textbook fixture that never reaches the branch. Now each mixes a group in with users, plus a principal that no longer exists, which is what makes the `!u` half of the guard load-bearing. The effective-access cache. Its eviction branch needed 256 distinct scopes to reach, so nothing ran it. Rather than build a fixture for that, the hand-rolled Map+eviction is replaced by createLru from src/effectiveAccess -- the identical cache, already extracted and tested for exactly this reason. Reuse over a second implementation, and the untestable branch stops existing. What remains is now covered: one scope is one entry however its node ids are ordered, a repeat is served rather than recomputed, and -- the one that matters -- a bumped sync version stops pre-crawl access being served, so a revoked grant cannot keep appearing in the matrix until the process restarts. --- app/api/src/matrix/inheritedAccess.js | 11 +- app/api/src/matrix/inheritedAccess.test.js | 268 +++++++++++++++++++++ app/api/stryker.matrix.config.json | 4 +- 3 files changed, 277 insertions(+), 6 deletions(-) diff --git a/app/api/src/matrix/inheritedAccess.js b/app/api/src/matrix/inheritedAccess.js index e68367bf2..545fc1b55 100644 --- a/app/api/src/matrix/inheritedAccess.js +++ b/app/api/src/matrix/inheritedAccess.js @@ -18,6 +18,7 @@ import { createHash } from 'crypto'; import { createParams } from '../db/sqlParams.js'; import * as db from '../db/connection.js'; import { effectiveAccessForNodes } from '../effectiveAccess/engine.js'; +import { createLru } from '../effectiveAccess/lru.js'; import { getSyncVersion } from '../lib/syncVersion.js'; import { resolveAttrExpr } from './attrExpr.js'; import { visibleKeyExpr } from './attributeCut.js'; @@ -28,9 +29,12 @@ import { GROUP_PRINCIPAL_TYPE } from '../lib/principalTypes.js'; // rows for a scope keyed by (sync-version, scope-hash). A crawl bumps the // sync-version → old keys become unreachable and age out. The subject filter is // applied per-request *after* the cache, so different subject scopes over the -// same resource scope share one entry. Bounded FIFO so memory can't grow. -const EFF_CACHE = new Map(); -const EFF_CACHE_MAX = 256; +// same resource scope share one entry. Bounded LRU so memory can't grow. +// createLru rather than a second hand-rolled Map+eviction: the identical cache +// already exists next door, with tests covering the recency and capacity +// behaviour this file's version had none of (its eviction branch needed 256 +// distinct scopes to reach, so nothing ever ran it). +const EFF_CACHE = createLru(256); async function cachedEffectiveAccess(nodeIds) { const hash = createHash('sha1').update([...nodeIds].sort().join(',')).digest('hex'); @@ -40,7 +44,6 @@ async function cachedEffectiveAccess(nodeIds) { const hit = EFF_CACHE.get(key); if (hit) return hit; const { rows } = await effectiveAccessForNodes(nodeIds); - if (EFF_CACHE.size >= EFF_CACHE_MAX) EFF_CACHE.delete(EFF_CACHE.keys().next().value); EFF_CACHE.set(key, rows); return rows; } diff --git a/app/api/src/matrix/inheritedAccess.test.js b/app/api/src/matrix/inheritedAccess.test.js index beca82224..ab96f7c43 100644 --- a/app/api/src/matrix/inheritedAccess.test.js +++ b/app/api/src/matrix/inheritedAccess.test.js @@ -15,6 +15,8 @@ vi.mock('./attributeCut.js', () => ({ visibleKeyExpr: vi.fn(() => 'pr."departmen import * as db from '../db/connection.js'; import { effectiveAccessForNodes } from '../effectiveAccess/engine.js'; import { resolveAttrExpr } from './attrExpr.js'; +import { getSyncVersion } from '../lib/syncVersion.js'; +import { GROUP_PRINCIPAL_TYPE } from '../lib/principalTypes.js'; import { buildInheritedFlatRows, buildInheritedRollupCounts, buildInheritedContextCounts, buildInheritedFoldCounts, explainInheritance, @@ -204,4 +206,270 @@ describe('explainInheritance', () => { expect(r.chain.map(c => c.id)).toEqual(['sub', 'focus']); expect(r.chain[0].isSource).toBe(true); }); + + // ── propagationScope: which ancestors actually GRANT the access ──────────── + // `reaches()` is the rule behind the "why does this person have access?" answer. + // Migration 038 defines propagationScope as self | descendants | selfAndDescendants, + // Azure-RBAC style: an assignment on the focus node itself only counts when its + // scope includes `self`, and one on an ancestor only counts when it includes + // `descendants`. Get it backwards and the explanation names a resource that does + // not grant the access, or hides the one that does — wrong in both directions and + // silent in both. + // + // Every scope value is asserted by WHICH ids come back, never by how many: a count + // cannot tell "the right two" from "a different two". + + const chainRow = (id, depth, scope, isSource, extra = {}) => ({ + id, depth, scope, isSource, + name: `n-${id}`, label: `l-${id}`, rolename: null, effect: null, ...extra, + }); + + it('applies each propagationScope by depth — self at the node, descendants above it', async () => { + // One fixture, five deliberately different rows. Each is the sole example of its + // case, so no two are interchangeable and a mutant that mixes them up changes the + // answer rather than landing on the same list. + db.query.mockResolvedValue({ rows: [ + chainRow('focus', 0, 'descendants', true), // on the node, but grants only BELOW it + chainRow('selfOnly', 1, 'self', true), // above the node, grants only at itself + chainRow('both', 2, 'selfAndDescendants', true, { rolename: 'Owner', effect: 'Allow' }), + chainRow('noScope', 3, null, true), // unset scope propagates (038 default) + chainRow('tooHigh', 4, 'descendants', false), // would reach, but carries no assignment + ] }); + + const r = await explainInheritance('focus', 'cap1', P1); + + // Only the two whose scope reaches the focus AND that carry an assignment. + expect(r.sources.map((s) => s.id)).toEqual(['both', 'noScope']); + // `tooHigh` is above the deepest source, so it is not part of the explanation. + expect(r.chain.map((c) => c.id)).toEqual(['noScope', 'both', 'selfOnly', 'focus']); + expect(r.chain.map((c) => c.isSource)).toEqual([true, true, false, false]); + }); + + it.each([ + ['self', 'the assignment applies at the node itself'], + ['selfAndDescendants', 'the assignment applies at the node and below'], + [null, 'an unset scope defaults to applying at the node'], + ])('counts a depth-0 assignment scoped %s as the source', async (scope) => { + // The negative direction is covered above (a depth-0 row scoped `descendants` is + // NOT a source). These are its counterparts: without them, a mutant that makes + // the depth-0 test always-false would leave that assertion still passing. + db.query.mockResolvedValue({ rows: [chainRow('focus', 0, scope, true)] }); + + const r = await explainInheritance('focus', 'cap1', P1); + + expect(r.sources.map((s) => s.id)).toEqual(['focus']); + expect(r.chain.map((c) => c.isSource)).toEqual([true]); + }); + + it('explains nothing above the focus when no assignment reaches it', async () => { + // No source => maxDepth falls back to 0, so the chain is the focus alone rather + // than the whole containment path. An off-by-one there would leak ancestors into + // an explanation that has nothing to explain. + db.query.mockResolvedValue({ rows: [ + chainRow('focus', 0, 'descendants', true), // grants below, not here + chainRow('parent', 1, 'self', true), // grants at itself, not here + ] }); + + const r = await explainInheritance('focus', 'cap1', P1); + + expect(r.sources).toEqual([]); + expect(r.chain.map((c) => c.id)).toEqual(['focus']); + }); + + it('reports the role and effect of each source, not just its identity', async () => { + db.query.mockResolvedValue({ rows: [ + chainRow('sub', 1, 'descendants', true, { rolename: 'Owner', effect: 'Allow' }), + chainRow('mg', 2, 'selfAndDescendants', true, { rolename: 'Reader', effect: 'Deny' }), + ] }); + + const r = await explainInheritance('focus', 'cap1', P1); + + // Distinct role AND effect per source: a mutant that reads either field off the + // wrong row changes one of these pairs. + expect(r.sources.map((s) => [s.id, s.role, s.effect])).toEqual([ + ['sub', 'Owner', 'Allow'], + ['mg', 'Reader', 'Deny'], + ]); + }); +}); + +// ── Group principals are containers, not holders ──────────────────────────── +// A group is how access is delivered, not someone who has it. Counting one as a +// holder double-counts the access — once for the group, once for each member who +// already appears through it — and puts a group's name in a list of people. Every +// builder therefore drops group principals, at three separate sites. +// +// Nothing exercised any of them before these tests: every existing fixture is a +// `User`, so the filters passed everything through and deleting them changed no +// result. Each test below mixes a group in with users and asserts WHICH ids +// survive, so a dropped or inverted filter changes the answer. +describe('group principals are excluded from holder rows and counts', () => { + const P2 = '44444444-4444-4444-4444-444444444444'; + const GRP = '55555555-5555-5555-5555-555555555555'; + const GHOST = '66666666-6666-6666-6666-666666666666'; + + // The effective-access cache is module-level and keyed by a hash of the scope's + // node ids, so each test scopes to its own node or it reads another test's rows. + const effRow = (node, principalId) => ({ + nodeId: node, resourceId: RES, displayName: 'Key Vault', resourceType: 'vault', + membershipType: 'Indirect', capabilityId: 'cap1', principalId, + }); + + it('keeps a group, and an unknown principal, out of the flat holder rows', async () => { + const N = 'aaaaaaa1-0000-0000-0000-000000000001'; + effectiveAccessForNodes.mockResolvedValue({ rows: [ + effRow(N, P1), effRow(N, P2), effRow(N, GRP), effRow(N, GHOST), + ] }); + db.query.mockImplementation((sql) => { + if (/FROM "Principals" WHERE id = ANY/.test(sql)) return Promise.resolve({ rows: [ + { id: P1, displayName: 'Alice', email: 'a@x', principalType: 'User', extendedAttributes: null }, + { id: P2, displayName: 'Bob', email: 'b@x', principalType: 'User', extendedAttributes: null }, + { id: GRP, displayName: 'Engineers', email: null, principalType: GROUP_PRINCIPAL_TYPE, extendedAttributes: null }, + // GHOST is deliberately absent: an effective-access row can outlive the + // principal it names, and reading properties off the missing row would throw. + ] }); + if (/FROM "Resources" r LEFT JOIN "Systems"/.test(sql)) return Promise.resolve({ rows: [{ id: N, systemId: 7, systemName: 'Azure' }] }); + return Promise.resolve({ rows: [] }); + }); + + const out = await buildInheritedFlatRows(makeP([N]), BUILT, 'principal', []); + + // Two users in, two rows out — by id, so "the right two" is distinguishable + // from "some other two". + expect(out.map((r) => r.memberId)).toEqual([P1, P2]); + expect(out.map((r) => r.memberDisplayName)).toEqual(['Alice', 'Bob']); + }); + + it('keeps a group out of the rolled-up per-group-value counts', async () => { + const N = 'aaaaaaa1-0000-0000-0000-000000000002'; + effectiveAccessForNodes.mockResolvedValue({ rows: [ + effRow(N, P1), effRow(N, P2), effRow(N, GRP), + ] }); + db.query.mockImplementation((sql) => { + // The group shares Engineering with P1, so counting it would read as 2 there + // while Sales stays at 1 — an asymmetry a per-group assertion can see. + if (/AS gv FROM "Principals"/.test(sql)) return Promise.resolve({ rows: [ + { id: P1, pt: 'User', gv: 'Engineering' }, + { id: P2, pt: 'User', gv: 'Sales' }, + { id: GRP, pt: GROUP_PRINCIPAL_TYPE, gv: 'Engineering' }, + ] }); + if (/FROM "Resources" r LEFT JOIN "Systems"/.test(sql)) return Promise.resolve({ rows: [{ id: N, systemId: 7, systemName: 'Azure' }] }); + return Promise.resolve({ rows: [] }); + }); + + const r = await buildInheritedRollupCounts(makeP([N]), BUILT, 'principal', 'department', []); + + const byGv = Object.fromEntries(r.counts.map((c) => [c.groupValue, c.directCount])); + expect(byGv).toEqual({ Engineering: 1, Sales: 1 }); + expect(Object.fromEntries(r.groupTotals.map((t) => [t.groupValue, t.total]))) + .toEqual({ Engineering: 1, Sales: 1 }); + }); + + it('keeps a group out of the frontier-context counts', async () => { + const N = 'aaaaaaa1-0000-0000-0000-000000000003'; + effectiveAccessForNodes.mockResolvedValue({ rows: [ + effRow(N, P1), effRow(N, P2), effRow(N, GRP), + ] }); + db.query.mockImplementation((sql) => { + if (/WITH RECURSIVE frontier/.test(sql)) return Promise.resolve({ rows: [ + { gv: 'ctx-A', pid: P1 }, { gv: 'ctx-A', pid: GRP }, { gv: 'ctx-B', pid: P2 }, + ] }); + if (/"principalType" AS pt FROM "Principals"/.test(sql)) return Promise.resolve({ rows: [ + { id: P1, pt: 'User' }, { id: P2, pt: 'User' }, { id: GRP, pt: GROUP_PRINCIPAL_TYPE }, + ] }); + if (/FROM "Resources" r LEFT JOIN "Systems"/.test(sql)) return Promise.resolve({ rows: [{ id: N, systemId: 7, systemName: 'Azure' }] }); + return Promise.resolve({ rows: [] }); + }); + + const r = await buildInheritedContextCounts(makeP([N]), BUILT, 'principal', ['ctx-A', 'ctx-B']); + + // ctx-A would read 2 if the group counted; ctx-B is 1 either way, which is what + // makes the pair discriminating rather than a single number that could be right + // for the wrong reason. + expect(Object.fromEntries(r.counts.map((c) => [c.groupValue, c.directCount]))) + .toEqual({ 'ctx-A': 1, 'ctx-B': 1 }); + }); +}); + +// ── The effective-access cache ────────────────────────────────────────────── +// Effective access is expensive, so it is cached per (sync-version, scope-hash). +// A cache that is wrong is silent in both directions: it serves access that is no +// longer granted, or misses constantly and the matrix crawls. None of this was +// exercised — the tests all used one node id and never called twice — so the key +// could have been built from anything and every test still passed. +// +// The cache is module-level, so these tests cannot reset it; each uses its own +// node ids and counts engine invocations instead. +describe('effective-access caching', () => { + const rowsFor = (node) => ({ rows: [{ + nodeId: node, resourceId: RES, displayName: 'Key Vault', resourceType: 'vault', + membershipType: 'Indirect', capabilityId: 'cap1', principalId: P1, + }] }); + + const stubDb = (node) => db.query.mockImplementation((sql) => { + if (/FROM "Principals" WHERE id = ANY/.test(sql)) return Promise.resolve({ rows: [{ id: P1, displayName: 'Alice', email: 'a@x', principalType: 'User', extendedAttributes: null }] }); + if (/FROM "Resources" r LEFT JOIN "Systems"/.test(sql)) return Promise.resolve({ rows: [{ id: node, systemId: 7, systemName: 'Azure' }] }); + return Promise.resolve({ rows: [] }); + }); + + it('treats the same scope as one entry however the node ids are ordered', async () => { + // Two nodes, requested in both orders. The key sorts them, so this is one + // scope and must cost one engine call — drop the sort and the second request + // computes the identical answer again, permanently halving the hit rate for + // any scope the database returns in a different order. + const A = 'bbbbbbb1-0000-0000-0000-00000000000a'; + const B = 'bbbbbbb1-0000-0000-0000-00000000000b'; + effectiveAccessForNodes.mockResolvedValue(rowsFor(A)); + stubDb(A); + + await buildInheritedFlatRows(makeP([A, B]), BUILT, 'principal', []); + await buildInheritedFlatRows(makeP([B, A]), BUILT, 'principal', []); + + expect(effectiveAccessForNodes).toHaveBeenCalledTimes(1); + }); + + it('serves a repeated scope from the cache instead of recomputing it', async () => { + const N = 'bbbbbbb1-0000-0000-0000-00000000000c'; + effectiveAccessForNodes.mockResolvedValue(rowsFor(N)); + stubDb(N); + + const first = await buildInheritedFlatRows(makeP([N]), BUILT, 'principal', []); + const second = await buildInheritedFlatRows(makeP([N]), BUILT, 'principal', []); + + expect(effectiveAccessForNodes).toHaveBeenCalledTimes(1); + // Same answer, not merely the same count — a cache that returned nothing on a + // hit would also satisfy the call count. + expect(second.map((r) => r.memberId)).toEqual(first.map((r) => r.memberId)); + expect(second).toHaveLength(1); + }); + + it('stops serving pre-crawl access once the sync version moves', async () => { + // The invalidation that matters: a crawl bumps the sync version, and access + // resolved before it must not be handed out afterwards. Without the version in + // the key, a revoked grant keeps appearing in the matrix until the process + // restarts — the cache would have no way to know the data underneath changed. + const N = 'bbbbbbb1-0000-0000-0000-00000000000d'; + effectiveAccessForNodes.mockResolvedValue(rowsFor(N)); + stubDb(N); + + getSyncVersion.mockResolvedValue(41); + await buildInheritedFlatRows(makeP([N]), BUILT, 'principal', []); + getSyncVersion.mockResolvedValue(42); + await buildInheritedFlatRows(makeP([N]), BUILT, 'principal', []); + + expect(effectiveAccessForNodes).toHaveBeenCalledTimes(2); + }); + + it('still answers when the sync version cannot be read', async () => { + // getSyncVersion failing must degrade to one shared bucket, not take the whole + // matrix down with it. + const N = 'bbbbbbb1-0000-0000-0000-00000000000e'; + effectiveAccessForNodes.mockResolvedValue(rowsFor(N)); + stubDb(N); + getSyncVersion.mockRejectedValue(new Error('no db')); + + const out = await buildInheritedFlatRows(makeP([N]), BUILT, 'principal', []); + + expect(out.map((r) => r.memberId)).toEqual([P1]); + }); }); diff --git a/app/api/stryker.matrix.config.json b/app/api/stryker.matrix.config.json index 843e5b868..169a411f7 100644 --- a/app/api/stryker.matrix.config.json +++ b/app/api/stryker.matrix.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR starts at 61, just under the first measurement, and only ratchets up -- raise it as survivors are killed.", + "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR ratchets up only: 61 at the first measurement, 65 once inheritedAccess.js went 65.14 -> 77.64 and the scope reached 67.02. Never lower it to make a red run green. Next targets by size of gap: scopeHistory.js (51.66), filterSql.js (56.52), rollupBuilders.js (61.67 at 100% coverage on every axis).", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -26,7 +26,7 @@ "thresholds": { "high": 90, "low": 70, - "break": 61 + "break": 65 }, "concurrency": 4, "timeoutMS": 60000, From 5adda99248aade2606c7c90ab635807d0a298f55 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 15:52:29 +0200 Subject: [PATCH 08/15] test: make the matrix filter and roll-up SQL fault-detecting Matrix scope 67.02% -> 72.91%, floor ratchets 65 -> 70. rollupBuilders.js 61.67% -> 96.67%; filterSql.js 56.52% -> 76.09%. Every kill verified by re-applying Stryker's exact reported span. rollupBuilders (21 of 23). All 23 survivors were WHERE-clause assembly, and they survived for one reason: the existing "omits the IN-clause when unscoped" tests assert `not.toMatch(/IN \(SELECT/)`, which passes just as happily on the broken `p."principalId" IN null` that dropping the guard actually produces. Asserting the ABSENCE of a string said nothing about what was there. These assert the whole span between two fixed anchors instead, so a lost WHERE keyword, a lost AND silently merging two conditions, an unasked-for condition, or the group-account exclusion vanishing all change the result. The 2 that remain are provably equivalent, not unexamined: both are the ' AND ' separator in builders whose bodies contain exactly one `where.push`, so the list can never hold two conditions and [x].join(sep) === x. Left visible rather than silenced with a disable comment, which would also hide the day a second condition makes them killable. Checking that claim is what caught the opposite case -- buildGroupTotalsSql looked identical but has TWO pushes (it also excludes group accounts over Principals), so its separator IS reachable and now has a test. filterSql (42 of 44 in the targeted clusters, then both remaining). Three things were entirely unmeasured: * Cross-entity context translation. A context filter names members of one kind while the query filters another, so it is rewritten -- principals expand down to identities, identities roll up to principals, systems match on systemId. Only the direct same-kind case was ever tested. A wrong rewrite still returns rows, they are simply the wrong people. Two of the mappings are exact mirror images differing only in which column is selected and which is matched, so each test asserts the direction rather than "IdentityMembers is involved", which is true of either. * Include vs exclude routing, for context conditions. Sending one down the wrong branch shows exactly the population the user asked to hide. The attribute tests could not see it -- the include assertion still matches when the clause is wrapped, because the wrapper goes around it. * The guard in collectContextIds. Nothing malformed was ever passed in. Two mutants needed inputs differing from a valid one in exactly ONE property to prove each check load-bearing, and both are shapes real JSON produces: a condition mis-tagged as an attribute while still carrying its context id, and an id wrapped in the single-element array a multi-select emits. The unusable-pairing table likewise had to cover every pairing rather than a sample -- Resource+Identity and Principal+System exist precisely because without them the `entity` half of two guards could be deleted with nothing failing. --- app/api/src/matrix/filterSql.test.js | 179 ++++++++++++++++++++++ app/api/src/matrix/rollupBuilders.test.js | 119 ++++++++++++++ app/api/stryker.matrix.config.json | 4 +- 3 files changed, 300 insertions(+), 2 deletions(-) diff --git a/app/api/src/matrix/filterSql.test.js b/app/api/src/matrix/filterSql.test.js index 46f3df88a..2e6b97669 100644 --- a/app/api/src/matrix/filterSql.test.js +++ b/app/api/src/matrix/filterSql.test.js @@ -133,3 +133,182 @@ describe('collectContextIds', () => { expect(ids).toHaveLength(1); }); }); + +// ── Context filters across entity types ───────────────────────────────────── +// A context filter names a set of members; the entity being filtered is often a +// different kind of thing, so the filter has to be translated. Every translation +// was unasserted: the tests above only ever put a Principal context on a Principal +// entity (the direct case) or an incompatible one (dropped with a warning), so all +// three cross-entity mappings could have been wrong or missing and nothing failed. +// +// A wrong translation here is the worst kind of silent: the query still runs and +// still returns rows, they are simply the wrong people. Two of the mappings are +// exact mirror images of each other — principal→identity and identity→principal +// differ only in which column is selected and which is matched — so each test +// asserts the DIRECTION, not merely that IdentityMembers is involved. +describe('context filters across entity types', () => { + const CTX_IDENTITY = 'a0000003-0000-0000-0000-000000000003'; + const CTX_SYSTEM = 'a0000004-0000-0000-0000-000000000004'; + const TYPES = new Map([ + [CTX_PRINCIPAL, 'Principal'], + [CTX_RESOURCE, 'Resource'], + [CTX_IDENTITY, 'Identity'], + [CTX_SYSTEM, 'System'], + ]); + + const buildFor = (entity, contextId) => { + const { params, bind } = createParams(); + const out = buildEntitySubquery({ + entity, + include: [{ kind: 'context', contextId, includeChildren: false }], + exclude: [], + validColumns: new Set(), + contextTypes: TYPES, + bind, + }); + return { ...out, params, norm: (out.sql || '').replace(/\s+/g, ' ').trim() }; + }; + + it('matches ids directly when the context already targets this entity', () => { + const out = buildFor('Principal', CTX_PRINCIPAL); + // No bridge table: the context members ARE the entities being filtered. + expect(out.norm).toContain('id IN ( SELECT "memberId" FROM "ContextMembers"'); + expect(out.norm).not.toContain('IdentityMembers'); + }); + + it('expands an Identity context down to its member principals', () => { + const out = buildFor('Principal', CTX_IDENTITY); + // Selects principalId, matches on identityId — the opposite of the roll-up + // below. Asserting both columns is what distinguishes the two directions; + // "uses IdentityMembers" is true of either and so proves nothing. + expect(out.norm).toContain('id IN (SELECT "principalId" FROM "IdentityMembers" WHERE "identityId" IN'); + }); + + it('rolls a Principal context up to the identities those principals belong to', () => { + const out = buildFor('Identity', CTX_PRINCIPAL); + expect(out.norm).toContain('id IN (SELECT "identityId" FROM "IdentityMembers" WHERE "principalId" IN'); + }); + + it('matches resources by system id for a System context', () => { + const out = buildFor('Resource', CTX_SYSTEM); + // Not `id IN` — a System context constrains which system a resource belongs + // to, so it filters on systemId, cast because ContextMembers.memberId is text. + expect(out.norm).toContain('"systemId"::text IN ('); + expect(out.norm).not.toContain('IdentityMembers'); + }); + + // Every entity/context pairing that has NO translation. This table has to cover + // each unusable pairing, not a sample of them: each mapping above is guarded by + // two conditions, and only a pairing that satisfies one of them proves the other + // is load-bearing. Resource+Identity and Principal+System are here for exactly + // that reason — without them, dropping the `entity` half of either guard changes + // nothing any test can see. + it.each([ + ['Principal', 'Resource', CTX_RESOURCE], + ['Identity', 'Resource', CTX_RESOURCE], + ['Resource', 'Principal', CTX_PRINCIPAL], + ['Resource', 'Identity', CTX_IDENTITY], + ['Principal', 'System', CTX_SYSTEM], + ['Identity', 'System', CTX_SYSTEM], + ])('drops a %s filter carrying an unusable %s context, with a warning', (entity, _kind, contextId) => { + const out = buildFor(entity, contextId); + // Dropped rather than ignored: silently returning every row would widen the + // filter to everything, which reads as "no matches were excluded". + expect(out.sql).toBeNull(); + expect(out.warnings).toHaveLength(1); + }); +}); + +// ── Include vs exclude routing ────────────────────────────────────────────── +// Every condition is pushed onto one of two lists by `target === 'inc'`, and the +// exclude list is the one wrapped in `IS NOT TRUE`. Send a condition down the +// wrong branch and the filter does the opposite of what was asked: the rows the +// user wanted hidden are the only ones they see. +// +// The existing tests could not detect that. The include test asserts +// `"department"::text IN` — which is still in the SQL when the condition is +// routed to exclude, because the wrapper goes AROUND it. Each direction has to +// assert the wrapper's presence AND its absence. +describe('include and exclude routing', () => { + const attrCond = { kind: 'attribute', field: 'department', values: ['Finance'] }; + const ctxCond = { kind: 'context', contextId: CTX_PRINCIPAL, includeChildren: false }; + + it('leaves an attribute include unwrapped', () => { + const out = buildPrincipal([attrCond], []); + expect(out.sql).toContain('"department"::text IN'); + expect(out.sql).not.toContain('IS NOT TRUE'); + }); + + it('wraps an attribute exclude so NULL attributes are kept', () => { + const out = buildPrincipal([], [attrCond]); + expect(out.sql).toContain('"department"::text IN'); + expect(out.sql).toContain('IS NOT TRUE'); + }); + + it('leaves a context include unwrapped', () => { + const out = buildPrincipal([ctxCond], []); + expect(out.sql).toContain('ContextMembers'); + expect(out.sql).not.toContain('IS NOT TRUE'); + }); + + it('wraps a context exclude', () => { + // Excluding by context was never exercised at all — only attributes were. + const out = buildPrincipal([], [ctxCond]); + expect(out.sql).toContain('ContextMembers'); + expect(out.sql).toContain('IS NOT TRUE'); + }); + + it('keeps both sides apart when a filter includes and excludes at once', () => { + const out = buildPrincipal( + [{ kind: 'attribute', field: 'department', values: ['Finance'] }], + [{ kind: 'attribute', field: 'jobTitle', values: ['Intern'] }], + ); + // The included field is bare; only the excluded one carries the wrapper. A + // routing bug that sent both the same way collapses this asymmetry. + expect(out.sql).toMatch(/"department"::text IN \([^)]*\) AND \("jobTitle"::text IN/); + expect(out.sql).toContain('IS NOT TRUE'); + expect(out.sql.match(/IS NOT TRUE/g)).toHaveLength(1); + }); +}); + +// ── collectContextIds only collects real context ids ──────────────────────── +// This prefetches the context types the filter will need. Its guard rejects +// anything that is not a context condition carrying a UUID string; loosen it and +// the caller looks up ids that cannot exist — or, with `c || c.kind`, dereferences +// a null condition. Nothing malformed was ever passed in, so the whole guard was +// unmeasured. +describe('collectContextIds rejects malformed conditions', () => { + it('collects only the well-formed context ids from a mixed list', () => { + const ids = collectContextIds({ + subject: { + include: [ + null, // no condition at all + { kind: 'attribute', field: 'department', values: ['Finance'] }, // not a context + { kind: 'context', contextId: 12345 }, // id is not a string + { kind: 'context', contextId: 'not-a-uuid' }, // string, but not an id + // Two inputs that differ from a valid one in EXACTLY one property, so + // each check is provably load-bearing. Both are shapes real JSON + // produces: a condition mis-tagged as an attribute while still carrying + // its context id, and an id wrapped in the single-element array that a + // multi-select control emits. + { kind: 'attribute', field: 'department', contextId: CTX_RESOURCE, values: [] }, + { kind: 'context', contextId: [CTX_RESOURCE] }, + { kind: 'context', contextId: CTX_PRINCIPAL }, // the only valid one + ], + exclude: [], + }, + resource: { include: [], exclude: [] }, + }); + // Exactly one survivor, named — four different ways of being malformed, so a + // guard that drops any one of its four checks admits a different id here. + expect(ids).toEqual([CTX_PRINCIPAL]); + }); + + it('ignores a side that is not an array', () => { + const ids = collectContextIds({ + subject: { include: 'nonsense', exclude: null }, + resource: { include: [{ kind: 'context', contextId: CTX_RESOURCE }], exclude: [] }, + }); + expect(ids).toEqual([CTX_RESOURCE]); + }); +}); diff --git a/app/api/src/matrix/rollupBuilders.test.js b/app/api/src/matrix/rollupBuilders.test.js index 588af86c2..a66a08a9c 100644 --- a/app/api/src/matrix/rollupBuilders.test.js +++ b/app/api/src/matrix/rollupBuilders.test.js @@ -1,5 +1,6 @@ import { describe, it, expect } from 'vitest'; import { buildRollupSql, buildRollupRolesSql, buildRolesAsRowsSql, buildGroupTotalsSql, buildRolesDrillSql } from './rollupBuilders.js'; +import { GROUP_PRINCIPAL_TYPE } from '../lib/principalTypes.js'; describe('buildRollupSql', () => { const base = { @@ -156,3 +157,121 @@ describe('buildRolesDrillSql', () => { expect(buildRolesDrillSql({ ...base, subjectSql: null })).not.toMatch(/WHERE/); }); }); + +// ── WHERE-clause assembly ─────────────────────────────────────────────────── +// Every builder assembles its WHERE clause the same way: a list of conditions, +// pushed conditionally, joined with AND, and prefixed with WHERE only if the list +// is non-empty. All of it was unasserted, because the existing "omits the +// IN-clause" tests check `not.toMatch(/IN \(SELECT/)` — and that passes just as +// happily on the broken `p."principalId" IN null` that dropping the guard +// produces. Asserting the ABSENCE of one string says nothing about what is +// actually there. +// +// These assert the whole span between two fixed anchors instead, so anything +// injected into or dropped from the WHERE slot changes the result: a lost WHERE +// keyword, a lost AND (silently merging two conditions into nonsense), a +// condition added when nothing should be filtered, or the group-account exclusion +// disappearing — which would quietly count group accounts as people. +describe('WHERE-clause assembly', () => { + const norm = (sql) => sql.replace(/\s+/g, ' ').trim(); + + // The generated text between `after` and `before`, whitespace-normalised. + // Omitting `before` takes everything to the end of the statement. + const between = (sql, after, before) => { + const t = norm(sql); + const a = t.indexOf(after); + expect(a, `anchor not found: ${after}`).toBeGreaterThan(-1); + const from = a + after.length; + const b = before ? t.indexOf(before, from) : t.length; + expect(b, `anchor not found: ${before}`).toBeGreaterThan(-1); + return t.slice(from, b).trim(); + }; + + const SUBJ = '(SELECT id FROM x)'; + const RES = '(SELECT id FROM y)'; + const base = { + attrExpr: 'u."department"', + subjectJoin: 'SJ', subjectIdExpr: 'p."principalId"', subjectIdForFilter: 'p."principalId"', + subjectNameExpr: 'NEXP', subjectTypeExpr: 'TEXP', + brMemberId: 'br."userId"', brJoin: 'BJ', + subjectTable: 'Identities', subjectAlias: 'i', + subjectSql: null, resourceSql: null, + }; + + // The group-account exclusion is the one condition that is always present, and + // the only builder that carries it. Built from the imported constant so a rename + // of the @odata.type moves the code and this expectation together. + const NOT_A_GROUP = + `(p."principalType" IS NULL OR p."principalType" != '${GROUP_PRINCIPAL_TYPE}')`; + + const CASES = [ + { + name: 'buildRollupSql', + build: (over) => buildRollupSql({ ...base, ...over }), + after: 'AND br."resourceId" = p."resourceId"', + before: 'GROUP BY', + unscoped: `WHERE ${NOT_A_GROUP} AND p."membershipType" = 'Direct'`, + scoped: `WHERE ${NOT_A_GROUP} AND p."membershipType" = 'Direct'` + + ` AND p."principalId" IN ${SUBJ} AND p."resourceId" IN ${RES}`, + over: { subjectSql: SUBJ, resourceSql: RES }, + }, + { + name: 'buildRollupRolesSql', + build: (over) => buildRollupRolesSql({ ...base, ...over }), + after: 'ON role.id = br."businessRoleId"', + before: 'GROUP BY', + unscoped: '', + scoped: `WHERE br."userId" IN ${SUBJ} AND br."resourceId" IN ${RES}`, + over: { subjectSql: SUBJ, resourceSql: RES }, + }, + { + name: 'buildRolesAsRowsSql', + build: (over) => buildRolesAsRowsSql({ ...base, ...over }), + after: 'ON role.id = br."businessRoleId"', + before: 'GROUP BY', + unscoped: '', + scoped: `WHERE p."principalId" IN ${SUBJ}`, + over: { subjectSql: SUBJ }, + }, + { + name: 'buildGroupTotalsSql', + build: (over) => buildGroupTotalsSql({ ...base, ...over }), + after: 'FROM "Identities" i', + before: 'GROUP BY', + unscoped: '', + scoped: `WHERE i.id IN ${SUBJ}`, + over: { subjectSql: SUBJ }, + }, + { + // The one builder that can hold TWO conditions without a resource scope: on + // the Principals table it also excludes group-shaped accounts. That is what + // makes its AND separator reachable at all — over Identities there is only + // ever one condition, so the join never uses it. + name: 'buildGroupTotalsSql over Principals', + build: (over) => buildGroupTotalsSql({ ...base, subjectTable: 'Principals', subjectAlias: 'p', ...over }), + after: 'FROM "Principals" p', + before: 'GROUP BY', + unscoped: `WHERE (p."principalType" IS NULL OR p."principalType" != '${GROUP_PRINCIPAL_TYPE}')`, + scoped: `WHERE (p."principalType" IS NULL OR p."principalType" != '${GROUP_PRINCIPAL_TYPE}')` + + ` AND p.id IN ${SUBJ}`, + over: { subjectSql: SUBJ }, + }, + { + name: 'buildRolesDrillSql', + build: (over) => buildRolesDrillSql({ ...base, ...over }), + after: '"vw_UserPermissionAssignmentViaBusinessRole" br SJ', + before: null, // the WHERE clause ends this statement + unscoped: '', + scoped: `WHERE p."principalId" IN ${SUBJ}`, + over: { subjectSql: SUBJ }, + }, + ]; + + it.each(CASES)('$name filters on nothing extra when no scope is supplied', (c) => { + expect(between(c.build({}), c.after, c.before)).toBe(c.unscoped); + }); + + it.each(CASES)('$name emits exactly the scoped conditions, joined and prefixed', (c) => { + expect(between(c.build(c.over), c.after, c.before)).toBe(c.scoped); + }); +}); diff --git a/app/api/stryker.matrix.config.json b/app/api/stryker.matrix.config.json index 169a411f7..cae3f074e 100644 --- a/app/api/stryker.matrix.config.json +++ b/app/api/stryker.matrix.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR ratchets up only: 61 at the first measurement, 65 once inheritedAccess.js went 65.14 -> 77.64 and the scope reached 67.02. Never lower it to make a red run green. Next targets by size of gap: scopeHistory.js (51.66), filterSql.js (56.52), rollupBuilders.js (61.67 at 100% coverage on every axis).", + "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR ratchets up only -- 61 -> 65 -> 70 as the scope went 63.44 -> 67.02 -> 72.91. Never lower it to make a red run green. rollupBuilders.js finishes at 96.67 with two survivors that are PROVABLY EQUIVALENT rather than unexamined: both are the ' AND ' separator in `where.join(' AND ')` inside buildRolesAsRowsSql and buildRolesDrillSql, and each of those has exactly one `where.push` in its body, so the list can never hold two conditions and `[x].join(sep) === x` for every sep. They are left visible instead of silenced: the floor sits below them, so they cost nothing, and a future second condition would make them killable again -- which a disable comment would hide. Next target: scopeHistory.js, still at 51.66 and now the only file under 68.", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -26,7 +26,7 @@ "thresholds": { "high": 90, "low": 70, - "break": 65 + "break": 70 }, "concurrency": 4, "timeoutMS": 60000, From 5486876b2ae073b2edfa5e0097eea9b6f6fcd561 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 16:46:01 +0200 Subject: [PATCH 09/15] test: make the as-of history reconstruction fault-detecting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit scopeHistory.js 51.66% -> 94.31%; matrix scope 72.91% -> 80.93%; floor 70 -> 78. 90 of its 102 undetected mutants killed, each verified by re-applying Stryker's exact reported span. This file re-expresses the matrix filters against reconstructed jsonb snapshots so the trend charts can answer "who had this access in March". Only the direct same-kind context case was ever exercised, which left unmeasured: both cross-entity translations, the entire exclude path, the value normalisation, the extendedAttributes path, and every guard. A wrong answer here is uniquely hard to notice. The live matrix can be checked against the system it mirrors; a historical figure has nothing to compare against, so it is believed by default. The sharpest gap was the query skeleton. Two of the four CTEs are built by one helper parameterised with the table to reconstruct, passed as a bare string -- `asofSurrogateCte('asof_principals', 'Principals', 'Principals')`. Nothing asserted which table each rebuilt, so swapping the pair would reconstruct principals from the Resources audit trail and produce SQL that still parses, still runs, and still returns rows. The test now pins each CTE to both places its table name appears: the `_history."tableName"` it replays and the live table it unions. Also closed: values are normalised before binding (empty ones dropped, the rest stringified, capped at 200) -- the fixture mixes three keepers of three different types against three ways of being empty, so letting one empty form through or dropping a falsy-but-real 0 changes the bound parameters; identifiers reaching an ext.-prefixed or plain column are validated before interpolation, which is what keeps a crafted field name out of the query text; and a missing subject or resource block is treated as no conditions rather than throwing. One correction along the way: the first version of the CTE assertion described a `WITH latest AS (SELECT DISTINCT ON …)` shape the file does not have. It failed, which is the point of writing the assertion against the real output rather than against what the code was assumed to emit. --- app/api/src/matrix/scopeHistory.test.js | 262 ++++++++++++++++++++++++ app/api/stryker.matrix.config.json | 4 +- 2 files changed, 264 insertions(+), 2 deletions(-) diff --git a/app/api/src/matrix/scopeHistory.test.js b/app/api/src/matrix/scopeHistory.test.js index 4a12f2222..e908d0288 100644 --- a/app/api/src/matrix/scopeHistory.test.js +++ b/app/api/src/matrix/scopeHistory.test.js @@ -103,3 +103,265 @@ describe('buildScopeAsofSql', () => { expect(warnings.some(w => /nope/.test(w))).toBe(true); }); }); + +// ── As-of scope conditions ────────────────────────────────────────────────── +// buildScopeAsofSql reconstructs the matrix scope at a past instant, so the same +// filter has to be re-expressed against a jsonb `state` snapshot rather than live +// tables. That re-expression was almost entirely unmeasured: the tests above use +// one Principal context on the subject block, which is the direct same-kind case, +// so every cross-entity translation, the exclude path, the value normalisation +// and the extendedAttributes path had no test at all. +// +// This is the history view. A wrong answer here is a trend line that shows access +// nobody had, or hides access somebody did — and unlike the live matrix there is +// nothing to compare it against. +describe('as-of scope conditions', () => { + const CTX_IDENTITY = 'a0000003-0000-0000-0000-000000000003'; + const CTX_SYSTEM = 'a0000004-0000-0000-0000-000000000004'; + const CTX_RESOURCE = 'a0000002-0000-0000-0000-000000000002'; + const TYPES = new Map([ + [CTX_PRINCIPAL, 'Principal'], + [CTX_RESOURCE, 'Resource'], + [CTX_IDENTITY, 'Identity'], + [CTX_SYSTEM, 'System'], + ]); + + const ctx = (contextId) => ({ kind: 'context', contextId, includeChildren: false }); + const norm = (sql) => (sql || '').replace(/\s+/g, ' '); + + const withSubject = (include, exclude = []) => build( + { ...EMPTY, subject: { include, exclude } }, TYPES); + const withResource = (include, exclude = []) => build( + { ...EMPTY, resource: { include, exclude } }, TYPES); + + describe('context translation across entity kinds', () => { + it('matches the reconstructed row id directly for a same-kind context', () => { + const out = withSubject([ctx(CTX_PRINCIPAL)]); + expect(norm(out.sql)).toContain(`(sp.state->>'id')::uuid IN (SELECT "memberId" FROM "ContextMembers"`); + expect(norm(out.sql)).not.toContain('IdentityMembers'); + }); + + it('expands an Identity context down to member principals', () => { + const out = withSubject([ctx(CTX_IDENTITY)]); + // Selects principalId, matches identityId. Asserting both columns is what + // pins the direction — "IdentityMembers appears" would hold either way. + expect(norm(out.sql)).toContain( + `(sp.state->>'id')::uuid IN (SELECT "principalId" FROM "IdentityMembers" WHERE "identityId" IN`); + }); + + it('matches a System context against the reconstructed systemId, not the row id', () => { + const out = withResource([ctx(CTX_SYSTEM)]); + expect(norm(out.sql)).toContain(`(sr.state->>'systemId') IN (SELECT "memberId" FROM "ContextMembers"`); + expect(norm(out.sql)).not.toContain(`(sr.state->>'id')::uuid IN`); + }); + + it.each([ + ['subject', 'Resource', CTX_RESOURCE], + ['subject', 'System', CTX_SYSTEM], + ['resource', 'Principal', CTX_PRINCIPAL], + ['resource', 'Identity', CTX_IDENTITY], + ])('drops a %s condition carrying an unusable %s context', (side, _kind, contextId) => { + // Each unusable pairing, not a sample: both branches above are guarded by + // two conditions, and only a pairing that satisfies one of them shows the + // other is doing work. + const out = side === 'subject' ? withSubject([ctx(contextId)]) : withResource([ctx(contextId)]); + expect(out.warnings.some((w) => /context condition dropped/.test(w))).toBe(true); + }); + + it('drops a context whose id is not a uuid', () => { + const out = build({ ...EMPTY, subject: { include: [ctx('not-a-uuid')], exclude: [] } }, + new Map([['not-a-uuid', 'Principal']])); + expect(out.warnings.some((w) => /context condition dropped/.test(w))).toBe(true); + }); + }); + + describe('include and exclude', () => { + const attr = { kind: 'attribute', field: 'department', values: ['Finance'] }; + + it('leaves an included condition unwrapped', () => { + const out = withSubject([attr]); + expect(norm(out.sql)).toContain(`(sp.state->>'department') IN`); + expect(out.sql).not.toContain('IS NOT TRUE'); + }); + + it('wraps an excluded condition so rows missing the attribute survive', () => { + // NOT (x IN (…)) is NULL when x is NULL, and NULL is falsy in WHERE — that + // would silently drop every row with an empty attribute from the history. + const out = withSubject([], [attr]); + expect(norm(out.sql)).toContain(`(sp.state->>'department') IN`); + expect(out.sql).toContain('IS NOT TRUE'); + }); + + it('wraps an excluded context condition too', () => { + const out = withSubject([], [ctx(CTX_PRINCIPAL)]); + expect(out.sql).toContain('IS NOT TRUE'); + }); + + it('keeps include and exclude apart within one block', () => { + const out = withSubject( + [{ kind: 'attribute', field: 'department', values: ['Finance'] }], + [{ kind: 'attribute', field: 'jobTitle', values: ['Intern'] }], + ); + expect(out.sql.match(/IS NOT TRUE/g)).toHaveLength(1); + expect(norm(out.sql)).toContain(`(sp.state->>'department') IN`); + expect(norm(out.sql)).toContain(`((sp.state->>'jobTitle') IN`); + }); + + it.each([ + ['null', null], + ['a string', 'nope'], + ['a number', 7], + ])('ignores %s where a condition object is expected', (_label, cond) => { + const out = withSubject([cond, { kind: 'attribute', field: 'department', values: ['Finance'] }]); + // The valid neighbour still lands, so this proves the bad one was skipped + // rather than the whole block abandoned. + expect(norm(out.sql)).toContain(`(sp.state->>'department') IN`); + expect(out.warnings).toHaveLength(0); + }); + + it('warns on a condition of an unrecognised kind', () => { + const out = withSubject([{ kind: 'sorcery', field: 'department' }]); + expect(out.warnings.some((w) => /unknown condition kind sorcery/.test(w))).toBe(true); + }); + }); + + describe('attribute values and fields', () => { + it('drops empty values and stringifies the rest', () => { + // Deliberately mixed and unequal: three keepers of three different types + // against three different ways of being empty, so a filter that lets one + // empty form through, or drops a falsy-but-real value like 0, changes the + // bound parameters. + const out = withSubject([{ + kind: 'attribute', field: 'department', + values: [null, 'Finance', '', 42, undefined, 0], + }]); + expect(out.params).toEqual(['Finance', '42', '0']); + }); + + it('returns no clause when every value is empty', () => { + const out = withSubject([{ kind: 'attribute', field: 'department', values: [null, '', undefined] }]); + expect(out.warnings.some((w) => /attribute condition dropped/.test(w))).toBe(true); + }); + + it('caps a condition at 200 values', () => { + const out = withSubject([{ + kind: 'attribute', field: 'department', + values: Array.from({ length: 250 }, (_, i) => `d${i}`), + }]); + expect(out.params).toHaveLength(200); + expect(out.params[199]).toBe('d199'); // the cap keeps the FIRST 200, not the last + }); + + it('reads an ext.-prefixed field out of the reconstructed extendedAttributes', () => { + const out = withSubject([{ kind: 'attribute', field: 'ext.costCentre', values: ['CC-1'] }]); + expect(norm(out.sql)).toContain(`(sp.state->'extendedAttributes'->>'costCentre') IN`); + }); + + it.each([ + ['ext.bad-key', 'an unsafe extendedAttributes key'], + ['bad-key', 'an unsafe column name'], + ['notAColumn', 'a column the entity does not have'], + ])('drops %s (%s)', (field) => { + // The identifier is interpolated into SQL rather than bound, so the guard + // is what stops a crafted field name reaching the query text. + const out = withSubject([{ kind: 'attribute', field, values: ['x'] }]); + expect(out.warnings.some((w) => /attribute condition dropped/.test(w))).toBe(true); + expect(norm(out.sql)).not.toContain('bad-key'); + }); + }); +}); + +// ── Query skeleton and guards ─────────────────────────────────────────────── +// The reconstruction query is assembled from four named CTEs plus the per-block +// WHERE fragments. None of the wiring was asserted: the CTE names and the tables +// they reconstruct are passed as bare strings, so a swapped pair would rebuild +// principals from the Resources audit trail and still produce runnable SQL. +describe('as-of query skeleton', () => { + const norm = (sql) => (sql || '').replace(/\s+/g, ' '); + + it('declares each reconstruction CTE once, over the table it belongs to', () => { + const out = build(EMPTY); + const names = [...norm(out.sql).matchAll(/(\w+) AS \(/g)].map((m) => m[1]); + // Order matters: each CTE is referenced by the ones after it. + expect(names.slice(0, 4)).toEqual(['asof_principals', 'asof_resources', 'asof_assign', 'asof_contains']); + // ...and each surrogate CTE reconstructs its own table: the audit rows it + // replays are selected by `_history."tableName"`, and the live rows it unions + // in come `FROM ""`. Swap the pair and principals get rebuilt from the + // Resources trail — SQL that still parses and still returns rows. + expect(norm(out.sql)).toContain(`asof_principals AS ( SELECT x."rowId" AS key`); + expect(norm(out.sql)).toContain(`asof_resources AS ( SELECT x."rowId" AS key`); + expect(norm(out.sql)).toMatch( + /asof_principals AS \(.*?h\."tableName" = 'Principals'.*?FROM "Principals" t.*?asof_resources AS \(/); + expect(norm(out.sql)).toMatch( + /asof_resources AS \(.*?h\."tableName" = 'Resources'.*?FROM "Resources" t.*?asof_assign AS \(/); + }); + + it('joins the principal conditions onto the standing group-account exclusion', () => { + const out = build({ ...EMPTY, subject: { + include: [{ kind: 'attribute', field: 'department', values: ['Finance'] }], exclude: [] } }); + // Two conditions, so the separator is load-bearing — asserting each one + // separately would pass even if they were run together into nonsense. + expect(norm(out.sql)).toContain( + `WHERE (sp.state->>'principalType' IS NULL OR sp.state->>'principalType' <> '#microsoft.graph.group')` + + ` AND (sp.state->>'department') IN`); + }); + + it('joins two conditions in one block with AND', () => { + const out = build({ ...EMPTY, subject: { include: [ + { kind: 'attribute', field: 'department', values: ['Finance'] }, + { kind: 'attribute', field: 'jobTitle', values: ['Analyst'] }, + ], exclude: [] } }); + expect(norm(out.sql)).toContain( + `(sp.state->>'department') IN ($1) AND (sp.state->>'jobTitle') IN ($2)`); + }); + + it('emits no resource WHERE at all when the resource block is unfiltered', () => { + const out = build(EMPTY); + // The resource scope selects from asof_resources with nothing between the + // alias and the closing paren — pinning the span catches anything injected + // into that slot, which asserting "no WHERE" would not. + expect(norm(out.sql)).toContain('FROM asof_resources sr )'); + }); + + it('emits the resource WHERE when the resource block is filtered', () => { + const out = build({ ...EMPTY, resource: { + include: [{ kind: 'attribute', field: 'resourceType', values: ['vault'] }], exclude: [] } }); + expect(norm(out.sql)).toContain(`FROM asof_resources sr WHERE (sr.state->>'resourceType') IN`); + }); + + it('treats a missing subject or resource block as no conditions', () => { + // `block?.include` — the caller may omit a side entirely rather than send an + // empty one, and dropping the optional chaining throws on the whole request. + const out = build({ rowType: 'principal' }); + expect(out.warnings).toEqual([]); + expect(norm(out.sql)).toContain('FROM asof_resources sr )'); + }); +}); + +describe('as-of condition guards', () => { + const norm = (sql) => (sql || '').replace(/\s+/g, ' '); + const subjectOnly = (cond) => build({ ...EMPTY, subject: { include: [cond], exclude: [] } }); + + it.each([ + ['a non-string field', { kind: 'attribute', field: 42, values: ['x'] }], + ['a non-array values', { kind: 'attribute', field: 'department', values: 'Finance' }], + ])('drops an attribute condition with %s', (_label, cond) => { + const out = subjectOnly(cond); + expect(out.warnings.some((w) => /attribute condition dropped/.test(w))).toBe(true); + expect(norm(out.sql)).not.toContain(`(sp.state->>'department') IN`); + }); + + it('drops a context whose type is unknown to the caller', () => { + // A valid uuid that the contextTypes map has never heard of: without the + // ctxType guard this builds a membership test against an undefined memberType. + const out = build({ ...EMPTY, subject: { + include: [{ kind: 'context', contextId: 'a0000009-0000-0000-0000-000000000009' }], exclude: [] } }, + new Map()); + expect(out.warnings.some((w) => /context condition dropped/.test(w))).toBe(true); + }); + + it('ignores a side that is not an array', () => { + const out = build({ ...EMPTY, subject: { include: 'nonsense', exclude: null } }); + expect(out.warnings).toEqual([]); + }); +}); diff --git a/app/api/stryker.matrix.config.json b/app/api/stryker.matrix.config.json index cae3f074e..ed78cce5d 100644 --- a/app/api/stryker.matrix.config.json +++ b/app/api/stryker.matrix.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR ratchets up only -- 61 -> 65 -> 70 as the scope went 63.44 -> 67.02 -> 72.91. Never lower it to make a red run green. rollupBuilders.js finishes at 96.67 with two survivors that are PROVABLY EQUIVALENT rather than unexamined: both are the ' AND ' separator in `where.join(' AND ')` inside buildRolesAsRowsSql and buildRolesDrillSql, and each of those has exactly one `where.push` in its body, so the list can never hold two conditions and `[x].join(sep) === x` for every sep. They are left visible instead of silenced: the floor sits below them, so they cost nothing, and a future second condition would make them killable again -- which a disable comment would hide. Next target: scopeHistory.js, still at 51.66 and now the only file under 68.", + "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR ratchets up only -- 61 -> 65 -> 70 -> 78 as the scope went 63.44 -> 67.02 -> 72.91 -> 80.93. Never lower it to make a red run green. rollupBuilders.js finishes at 96.67 with two survivors that are PROVABLY EQUIVALENT rather than unexamined: both are the ' AND ' separator in `where.join(' AND ')` inside buildRolesAsRowsSql and buildRolesDrillSql, and each of those has exactly one `where.push` in its body, so the list can never hold two conditions and `[x].join(sep) === x` for every sep. They are left visible instead of silenced: the floor sits below them, so they cost nothing, and a future second condition would make them killable again -- which a disable comment would hide. scopeHistory.js followed, 51.66 -> 94.31. Remaining targets, both still on their first measurement: contextRollup.js (68.80) and attributeCut.js (70.93).", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -26,7 +26,7 @@ "thresholds": { "high": 90, "low": 70, - "break": 70 + "break": 78 }, "concurrency": 4, "timeoutMS": 60000, From d762e0af237fb210380d11811f5a2818ffd0d92a Mon Sep 17 00:00:00 2001 From: Taeke Date: Wed, 19 Aug 2026 12:16:10 +0200 Subject: [PATCH 10/15] docs: give this PR its own changelog fragment Same reason as the parent branch: bump-version.yml merges every changes/*.md into CHANGES.md and then deletes them, so a stack sharing one cumulative fragment republishes the lower PRs' bullets each time a higher one merges. This branch's four bullets move to a file named after it; the shared fragment goes back to carrying only the PowerShell work it belongs to. --- changes/matrix-mutation-scope.md | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 changes/matrix-mutation-scope.md diff --git a/changes/matrix-mutation-scope.md b/changes/matrix-mutation-scope.md new file mode 100644 index 000000000..02cc07b4d --- /dev/null +++ b/changes/matrix-mutation-scope.md @@ -0,0 +1,4 @@ +- Extended the weekly fault-detection run to the matrix layer — the code that decides which access appears once inheritance, context rollups and attribute cuts are applied. It scores lowest of everything measured so far: 63% of injected faults are caught, against a line coverage of 94%. One file is caught by 100% of lines, branches and functions and still misses nearly four in ten faults, which is the clearest evidence yet that a coverage percentage is not a statement about whether bugs would be noticed. +- Closed the largest gaps this found in how inherited access is explained and counted. The rule deciding which parent resource actually grants a person's access — whether a permission applies at a resource, below it, or both — had no test at all, so any of it could have been backwards and every test still passed; the "why does this person have access" panel could have named the wrong resource, or hidden the real one. Group accounts were also never checked against the rule that excludes them from holder lists, so a group could have been counted as a person who has access, inflating counts and putting a group's name in a list of people. And the cache in front of these answers is now proven to stop serving access from before a crawl, rather than continuing to show permissions that had since been revoked. +- Proved out the filter and roll-up logic behind the matrix. Filters that select people by a context — a department, a business process — are translated when the thing being filtered is a different kind of thing to the context's members; none of those translations was tested, and two of them are mirror images, so either could have been pointing the wrong way and silently returning the wrong population. The routing that decides whether a condition includes or excludes was also untested for context filters, which is the difference between hiding a group of people and showing only them. Roll-up count queries are now checked for the whole condition they build rather than for one fragment of it, so a lost keyword or a dropped exclusion of group accounts is caught. +- Proved out the history view — the reconstruction of who had access at a past date, which powers the trend charts. Almost none of it was checked: the same filters the live matrix uses have to be re-expressed against reconstructed snapshots, and every cross-kind translation, the exclude path, the handling of empty or non-text filter values, and the custom-attribute path had no test. Two of the four reconstruction steps are also passed the table they rebuild as a plain name, so a swapped pair would have rebuilt people from the resource audit trail and still produced a working query and a plausible-looking chart. A wrong number here is uniquely hard to notice, because there is nothing to compare a historical figure against. From 5d19e4a15ea495fd8f32de21c38247dbd136ddf8 Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 17:59:48 +0200 Subject: [PATCH 11/15] ci: open a UI mutation scope, and stop mutating string literals Two changes, one prompted by the other. STRING MUTATION OFF, matching the three API configs that predate this work. My matrix and hooks configs had it on and I argued my way into each without checking what the repo already did -- all three older configs exclude StringLiteral and ObjectLiteral. The cost is real and recurring: string/object mutants were 20-43% of every scope measured (42.7% of auth, 25.9% of matrix), paid on every weekly run, while the yield is uneven -- on prose it is near zero (68 of 72 survivors on permissions.js) and where it does pay it mostly pays once. The assertions written to kill them stay and keep earning; only the weekly re-measuring stops. The matrix scope re-measures at 83.79% with them off, up from 80.93%, because those mutants were being killed at a LOWER rate than the rest -- not an improvement, a smaller denominator. Floor 78 -> 81. contextRollup.js moves most (68.80 -> 82.76), which says most of what was unkilled in it was text. That number is a re-run, not arithmetic. I first derived it by dropping string mutants from the existing report, got 83.39, and said excluding a mutator cannot change another mutant's verdict. The real run says 83.79, and up to 3 points out per file, so that reasoning was wrong -- exclusion does not simply subtract. stryker.pilot.config.json is deliberately left with strings on: it predates this work and carries an explicit argument that 'admin.auth' is the permission a hook actually checks, so those literals are behaviour. Not mine to overrule silently. THE UI SCOPE. Four hooks that decide which rows and which access a reviewer sees: the search/filter/pagination behind every list page, the walk that expands a group into the groups it belongs to (how inherited access appears at all), the matrix's data loading and debounce, and the persisted row order. file lines branch mutation useMatrix.js 97.3% 78.5% 41.22% useEntityPage.js 80.2% 71.7% 57.19% useNestedGroupExpand.js 98.5% 66.7% 60.00% useMatrixRowOrder.js 100.0% 87.5% 91.23% SCOPE 90.3% 75.5% 54.27% 54.27% against 90.3% line coverage is the widest gap measured here, and useMatrix.js at 97.3% line / 41.22% mutation is a 56-point spread -- more than double the previous worst (effectiveAccess/engine.js, 93 -> 69). The UI was the part of this codebase we knew least about and the answer is the least reassuring. Floor starts at 52. The test include is wider than the four hooks' own tests, and the width was measured rather than assumed: their own tests alone reach 70.63% branch, the full UI suite 75.46%, and the chosen include 75.46% with identical uncovered lines -- so the narrowing costs nothing while the obvious include would have manufactured survivors no test could kill. Job timeout 45 -> 60 minutes. 891 UI mutants took ~24 minutes locally against ~4 for 1,122 API mutants: a React render under jsdom per mutant is an order of magnitude more expensive. A UI scope that starts crowding that budget wants splitting into a second config rather than a longer timeout. --- .ci/js-mutation-scope-baseline.json | 4 --- .github/workflows/js-mutation.yml | 12 +++++-- app/api/stryker.matrix.config.json | 10 ++++-- app/ui/package.json | 5 +-- app/ui/stryker.hooks.config.json | 36 +++++++++++++++++++ app/ui/vitest.stryker.hooks.config.js | 51 +++++++++++++++++++++++++++ 6 files changed, 107 insertions(+), 11 deletions(-) create mode 100644 app/ui/stryker.hooks.config.json create mode 100644 app/ui/vitest.stryker.hooks.config.js diff --git a/.ci/js-mutation-scope-baseline.json b/.ci/js-mutation-scope-baseline.json index dd396eddf..a3b13c9f5 100644 --- a/.ci/js-mutation-scope-baseline.json +++ b/.ci/js-mutation-scope-baseline.json @@ -357,13 +357,9 @@ "app/ui/src/hooks/useDebouncedValue.js", "app/ui/src/hooks/useDocsUrl.js", "app/ui/src/hooks/useElapsedTimer.js", - "app/ui/src/hooks/useEntityPage.js", "app/ui/src/hooks/useExpandableGraph.js", "app/ui/src/hooks/useFeatures.js", "app/ui/src/hooks/useFetch.js", - "app/ui/src/hooks/useMatrix.js", - "app/ui/src/hooks/useMatrixRowOrder.js", - "app/ui/src/hooks/useNestedGroupExpand.js", "app/ui/src/hooks/usePermissions.js", "app/ui/src/hooks/usePersistedState.js", "app/ui/src/hooks/useRecentChanges.js", diff --git a/.github/workflows/js-mutation.yml b/.github/workflows/js-mutation.yml index b202bc4e2..24d4bb2a4 100644 --- a/.github/workflows/js-mutation.yml +++ b/.github/workflows/js-mutation.yml @@ -19,7 +19,7 @@ # time; that is the deliberate trade for the runtime, same as ps-mutation.yml. # test/ci-scripts/test-gate-wiring.sh enforces this for every *-mutation.yml. # -# SCOPE IS SMALL AND THAT IS THE POINT OF PUBLISHING IT. Twenty API/UI files are +# SCOPE IS SMALL AND THAT IS THE POINT OF PUBLISHING IT. 24 API/UI files are # mutation-tested out of ~410 eligible; the remaining backlog is counted in # .ci/js-mutation-scope-baseline.json and guarded by app/api/src/mutationScope. # guard.test.js, which fails when new code is added without a decision. Publishing @@ -59,7 +59,12 @@ jobs: mutation: name: 'Mutation: ${{ matrix.scope.label }}' runs-on: ubuntu-latest - timeout-minutes: 45 + # 60, not 45, because the UI scopes are an order of magnitude more expensive + # per mutant than the API ones: every mutant re-renders React under jsdom. + # Measured locally -- 891 UI mutants took ~24 minutes, 1,122 API mutants ~4. + # A scope that starts crowding this budget wants splitting into a second + # config rather than a longer timeout. + timeout-minutes: 60 strategy: # A scope that lands below its floor must not cancel the other three. The # point of the weekly run is a complete picture; losing three numbers to one @@ -71,7 +76,8 @@ jobs: - { pkg: api, name: effectiveaccess, label: 'API effective access' } - { pkg: api, name: accountlinking, label: 'API account linking' } - { pkg: api, name: matrix, label: 'API matrix' } - - { pkg: ui, name: pilot, label: 'UI' } + - { pkg: ui, name: pilot, label: 'UI permissions + filter' } + - { pkg: ui, name: hooks, label: 'UI matrix + list hooks' } steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: diff --git a/app/api/stryker.matrix.config.json b/app/api/stryker.matrix.config.json index ed78cce5d..2259672c8 100644 --- a/app/api/stryker.matrix.config.json +++ b/app/api/stryker.matrix.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: everything enabled -- deliberately NOT copying the auth config's StringLiteral/ObjectLiteral exclusions, and the code decided it rather than the assumption. These are SQL builders and app/api/CLAUDE.md notes the unit mocks are SQL-blind, which suggested mutated SQL text would be unkillable; but the tests here assert on the emitted SQL directly (`expect(sql).toContain(\"COUNT(*) FILTER (WHERE t.governed)...\")`), so a mutated literal changes text a test is pinning and dies. That is the opposite of permissions.js, where the strings were prose and 68 of 72 mutants survived. Carry the caveat with the number: pinning SQL text proves it is UNCHANGED, not that the query returns the right rows. Query correctness is the contract tests' job and a high score here does not cover it. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR ratchets up only -- 61 -> 65 -> 70 -> 78 as the scope went 63.44 -> 67.02 -> 72.91 -> 80.93. Never lower it to make a red run green. rollupBuilders.js finishes at 96.67 with two survivors that are PROVABLY EQUIVALENT rather than unexamined: both are the ' AND ' separator in `where.join(' AND ')` inside buildRolesAsRowsSql and buildRolesDrillSql, and each of those has exactly one `where.push` in its body, so the list can never hold two conditions and `[x].join(sep) === x` for every sep. They are left visible instead of silenced: the floor sits below them, so they cost nothing, and a future second condition would make them killable again -- which a disable comment would hide. scopeHistory.js followed, 51.66 -> 94.31. Remaining targets, both still on their first measurement: contextRollup.js (68.80) and attributeCut.js (70.93).", + "_comment": "The matrix layer -- the code that decides which access appears in the matrix once inheritance, context rollups and attribute cuts are applied. Chosen as the next scope because it is the closest analogue of the two worst surprises measured so far: src/effectiveAccess/engine.js read 93% line coverage and scored 69% under mutation, src/accountlinking/classifier.js read 97% and scored 68%. Six of these eight files sit at 100% line coverage with their own dedicated test file, which is the same profile, and inheritedAccess.js decides whether a grant is shown as held directly or through a group -- a wrong answer there is silent in both directions and is exactly what a reviewer reads off the matrix. Run: cd app/api && npm run test:mutation:matrix. TEST INCLUDE IS WIDER THAN src/matrix -- see vitest.stryker.matrix.config.js for why: most of these modules are also driven through the route layer, and excluding those tests would manufacture false survivors. MUTATOR SET: StringLiteral and ObjectLiteral excluded, matching every other API config. They were enabled for the first measurements and did pay here -- these tests assert on the emitted SQL, so a mutated literal changes text a test pins and dies, which is the opposite of permissions.js where the strings were prose and 68 of 72 survived. But that yield is mostly one-off, while the cost recurs on every weekly run: string/object mutants were 291 of 1,122 here, 26% of the work. The assertions written to kill them stay and keep earning; only the weekly re-measuring of them stops. Note the score RISES on exclusion, 80.93 -> 83.79, because those mutants were killed at a lower rate than the rest -- read the floor below as measured over a smaller mutant set, not as an improvement. Two files move most: contextRollup.js 68.80 -> 82.76 and attributeCut.js 70.93 -> 74.24, which says most of what was unkilled in them was text. 83.79 is a re-run, not arithmetic on the old report: deriving it by dropping string mutants from the previous report gave 83.39, and up to 3 points out per file, so excluding a mutator does not simply subtract its mutants. Carry this caveat regardless: pinning SQL text proves it is unchanged, not that the query returns the right rows. Query correctness is the contract tests' job. MEASURED 63.44% AT A SUITE LINE COVERAGE OF 94.3% -- the widest coverage-to-fault-detection gap recorded in this repo, and the reason this scope was picked. rollupBuilders.js is the sharpest case: 100% of lines, 100% of branches and 100% of functions, and 38% of injected faults still go unnoticed. THE INCLUDE SET IS NOT THE CAUSE: running the full API suite with coverage limited to src/matrix reproduces these per-file figures exactly (94.26% lines / 76.24% branch, same uncovered lines), so the 103 no-coverage mutants are real gaps, not tests this config left out. FLOOR ratchets up only -- 61 -> 65 -> 70 -> 78 as the scope went 63.44 -> 67.02 -> 72.91 -> 80.93 with string mutation on, then 81 against the 83.79 the same tests score with it off. Never lower it to make a red run green. rollupBuilders.js finishes at 96.67 with two survivors that are PROVABLY EQUIVALENT rather than unexamined: both are the ' AND ' separator in `where.join(' AND ')` inside buildRolesAsRowsSql and buildRolesDrillSql, and each of those has exactly one `where.push` in its body, so the list can never hold two conditions and `[x].join(sep) === x` for every sep. They are left visible instead of silenced: the floor sits below them, so they cost nothing, and a future second condition would make them killable again -- which a disable comment would hide. scopeHistory.js followed, 51.66 -> 94.31. Remaining targets, both still on their first measurement: contextRollup.js (68.80) and attributeCut.js (70.93).", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -23,10 +23,16 @@ "src/matrix/rollupBuilders.js", "src/matrix/scopeHistory.js" ], + "mutator": { + "excludedMutations": [ + "StringLiteral", + "ObjectLiteral" + ] + }, "thresholds": { "high": 90, "low": 70, - "break": 78 + "break": 81 }, "concurrency": 4, "timeoutMS": 60000, diff --git a/app/ui/package.json b/app/ui/package.json index b2b2488f6..9a0a61eb6 100644 --- a/app/ui/package.json +++ b/app/ui/package.json @@ -17,8 +17,9 @@ "test:e2e:sql": "bash scripts/e2e-sql.sh", "test:e2e:headed": "npx playwright test --headed", "test:e2e:ui": "npx playwright test --ui", - "test:mutation": "npm run test:mutation:pilot", - "test:mutation:pilot": "stryker run stryker.pilot.config.json" + "test:mutation": "npm run test:mutation:pilot && npm run test:mutation:hooks", + "test:mutation:pilot": "stryker run stryker.pilot.config.json", + "test:mutation:hooks": "stryker run stryker.hooks.config.json" }, "dependencies": { "@azure/msal-browser": "^5.18.0", diff --git a/app/ui/stryker.hooks.config.json b/app/ui/stryker.hooks.config.json new file mode 100644 index 000000000..bf9109a85 --- /dev/null +++ b/app/ui/stryker.hooks.config.json @@ -0,0 +1,36 @@ +{ + "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", + "_comment": "The first real UI scope beyond the two-file pilot, and it is deliberately the hooks rather than the components: these four decide WHICH rows and WHICH access a reviewer is shown, and each is wrong in a way nobody can see from the screen. useEntityPage owns the search, filter, tag and pagination state behind every list page -- a wrong filter silently shows a different population, and it is the least covered of the four (80.2% line). useNestedGroupExpand walks a group to the groups it is a member of, up to MAX_NEST_LEVEL, which is how inherited access appears at all; stop one level short and access simply is not displayed. useMatrix gates whether the matrix fetches anything and debounces the filter, so a fault there shows stale or empty results that look like 'no access'. useMatrixRowOrder persists custom row order under a version key -- forget to invalidate on a version bump and the user sees an order the current sort logic no longer produces. Run: cd app/ui && npm run test:mutation:hooks. TEST INCLUDE IS WIDER THAN THE FOUR HOOKS' OWN TESTS and the width was measured, not guessed -- see vitest.stryker.hooks.config.js: the hooks' own tests alone reach 70.63% branch, the full UI suite 75.46%, and the chosen include 75.46% with identical uncovered lines. MUTATOR SET: StringLiteral and ObjectLiteral excluded, matching every API config. String mutation is expensive -- it was 20-43% of the mutants in each scope measured so far -- and that cost is paid on every weekly run forever, while the yield is uneven: on prose it is near zero (68 of 72 survivors on permissions.js), and where it does pay it usually pays once. Some strings here ARE behaviour (a localStorage key, a version suffix, a query-parameter name) and those mutants go unmeasured as a result; that is the accepted trade, not an oversight. If a specific string turns out to carry a decision worth pinning, write the assertion for it rather than re-enabling the mutator wholesale. KNOWN FLAKINESS: the UI suite is stable on its own (157 files, 1273 tests green) but produced 2 then 6 different failures across two runs under coverage instrumentation at full parallelism. If this run reports scattered timeouts rather than a consistent set, lower `concurrency` before believing the number. MEASURED 54.27% AGAINST 90.3% LINE COVERAGE -- a 36-point gap, the widest of any scope here, and useMatrix.js alone reads 97.3% line and 41.22% mutation. That 56-point spread is more than double the previous worst (effectiveAccess/engine.js at 93 -> 69). The UI was the part of this codebase we knew least about and the answer is the least reassuring. RUNTIME: 891 mutants took ~24 minutes locally, against ~4 minutes for the 1,122-mutant API matrix scope -- jsdom plus a React render per mutant is roughly an order of magnitude more expensive per mutant. That is why js-mutation.yml allows 60 minutes per scope. Adding files here costs real weekly minutes; prefer a second config over growing this one past the budget. FLOOR starts at 52, just under the first measurement, and only ratchets up.", + "packageManager": "npm", + "testRunner": "vitest", + "vitest": { + "configFile": "vitest.stryker.hooks.config.js" + }, + "reporters": [ + "clear-text", + "json" + ], + "jsonReporter": { + "fileName": "reports/stryker-ui-hooks.json" + }, + "mutate": [ + "src/hooks/useEntityPage.js", + "src/hooks/useMatrix.js", + "src/hooks/useMatrixRowOrder.js", + "src/hooks/useNestedGroupExpand.js" + ], + "mutator": { + "excludedMutations": [ + "StringLiteral", + "ObjectLiteral" + ] + }, + "thresholds": { + "high": 90, + "low": 70, + "break": 52 + }, + "concurrency": 4, + "timeoutMS": 60000, + "disableTypeChecks": false +} diff --git a/app/ui/vitest.stryker.hooks.config.js b/app/ui/vitest.stryker.hooks.config.js new file mode 100644 index 000000000..74bb25ede --- /dev/null +++ b/app/ui/vitest.stryker.hooks.config.js @@ -0,0 +1,51 @@ +import { defineConfig } from 'vite'; +import react from '@vitejs/plugin-react'; +import path from 'path'; + +// Vitest config used only by the hooks mutation run (stryker.hooks.config.json). +// +// Standalone rather than spreading vite.config.js, for the same reasons as +// vitest.stryker.config.js: Stryker sandboxes app/ui alone, so the normal test +// `include` reaching ../../tools/crawlers/** resolves nothing there and aborts the +// run before a mutant is evaluated, and the tailwind plugin plus the '@crawlers' +// alias are build concerns that buy the mutation run nothing. +// +// WIDER THAN THE FOUR HOOKS' OWN TESTS, AND MEASURED RATHER THAN GUESSED. The +// obvious include — the four *.test.jsx files next to the hooks — leaves branches +// unexercised that the component mount tests do reach, and every one of those +// would have come back a SURVIVOR that no test could ever kill. Measured before +// choosing: coverage of the four hooks under the hooks' own tests alone is +// 70.63% branch, under the full UI suite 75.46%, and under the list below 75.46% +// — identical to the full suite, with the same uncovered lines. So the narrowing +// costs nothing. +// +// EntityListPage.mount.test.jsx drives useEntityPage through the real list page +// MatrixView.mount.test.jsx drives useMatrixRowOrder + useNestedGroupExpand +// App.mount.test.jsx the only test that mounts the useMatrix caller +// nestedRows.helpers.test.js the row builder that consumes the expand cache +// +// pageRegistry.test.jsx is deliberately absent: it was measured too and moved none +// of these numbers, so it cannot be any mutant's only killer here. +// +// Re-measure this comparison when adding a hook to the `mutate` list — the answer +// is per-file, not a property of the directory. + +export default defineConfig({ + plugins: [react()], + resolve: { + alias: { '@ui': path.resolve(import.meta.dirname, 'src') }, + }, + test: { + include: [ + 'src/hooks/useMatrix.test.jsx', + 'src/hooks/useNestedGroupExpand.test.jsx', + 'src/hooks/useEntityPage.test.jsx', + 'src/hooks/useMatrixRowOrder.test.js', + 'src/components/EntityListPage.mount.test.jsx', + 'src/components/MatrixView.mount.test.jsx', + 'src/App.mount.test.jsx', + 'src/components/matrix/nestedRows.helpers.test.js', + ], + exclude: ['**/node_modules/**'], + }, +}); From 2e50e4db6ee64a35381d7c11d9696f76fbea368d Mon Sep 17 00:00:00 2001 From: Taeke Date: Tue, 18 Aug 2026 21:07:09 +0200 Subject: [PATCH 12/15] test: close the matrix data-loading gaps, and give the UI run a real budget useMatrix.js 41.22% -> 69.18%; the UI hooks scope 54.27% -> 65.18%; floor 52 -> 63. Its unreached mutants fall from 28 to 7. The response normaliser was the bulk of it. Seventeen fields, each read as `body.x || `, and that one expression carries two silent failures: lose the default and undefined reaches the renderer, drawing an empty matrix that looks exactly like "this person has no access"; let the default win over a present value and real data disappears the same way. The existing test asserted two of the seventeen. Now the whole object is asserted twice -- once from a payload where every field carries a DIFFERENT value, so a mapper reading the wrong key produces a different object rather than coincidentally the same one (booleans true and maxDepth 4 precisely because those are not the defaults), and once from a minimal payload pinning every default. The four reference fetches each have their own catch and none had ever been made to fail. They are not the same fallback, which is the point: tags fail CLOSED (empty map, because null means "still loading" to every consumer) while the has-data check fails OPEN. That asymmetry is deliberate -- hasData=false routes the user to an empty-state screen telling them to import data, and a transient failure must not do that to a tenant whose data is fine. RUNTIME IS THE REAL FINDING. This scope took 24 minutes when opened and 58 after six tests were added, against a 60-minute job budget. That is not a regression: a mutant costs one run of every test that covers it, so better-covered code is slower to mutate, and a React render under jsdom makes each run an order of magnitude dearer than an API one. The budget has to hold a scope's future cost, not today's, and CI hardware is slower than the machine that measured this -- so the job timeout goes to 120. The generous ceiling is not licence to grow into it: the config now says plainly that this scope is at its practical limit and the next hook belongs in a second config, split along mount-test lines so each config's mutants run fewer of them rather than merely dividing the same work. Also worth recording: verifying each kill individually the way the API work did does not scale here. 164 mutants at ~18s of jsdom startup each is ~50 minutes, it timed out partway, and it left the source mutated on disk -- caught and restored, but the technique is a hazard at this cost. The Stryker re-run is both authoritative and far cheaper, because it reuses test-runner processes instead of paying startup per mutant. --- .github/workflows/js-mutation.yml | 17 ++- app/ui/src/hooks/useMatrix.test.jsx | 169 ++++++++++++++++++++++++++++ app/ui/stryker.hooks.config.json | 4 +- 3 files changed, 182 insertions(+), 8 deletions(-) diff --git a/.github/workflows/js-mutation.yml b/.github/workflows/js-mutation.yml index 24d4bb2a4..08dd15b56 100644 --- a/.github/workflows/js-mutation.yml +++ b/.github/workflows/js-mutation.yml @@ -59,12 +59,17 @@ jobs: mutation: name: 'Mutation: ${{ matrix.scope.label }}' runs-on: ubuntu-latest - # 60, not 45, because the UI scopes are an order of magnitude more expensive - # per mutant than the API ones: every mutant re-renders React under jsdom. - # Measured locally -- 891 UI mutants took ~24 minutes, 1,122 API mutants ~4. - # A scope that starts crowding this budget wants splitting into a second - # config rather than a longer timeout. - timeout-minutes: 60 + # 120, because the UI scopes are an order of magnitude more expensive per + # mutant than the API ones: every mutant re-renders React under jsdom. + # Measured locally -- the 891-mutant UI hooks scope took 24 minutes when first + # opened and 58 after six tests were added, against ~4 minutes for 1,122 API + # mutants. A mutant costs one run of every test covering it, so the better the + # tests get the slower the run: the budget has to hold the scope's FUTURE cost, + # not today's, and CI hardware is slower than the machine that measured this. + # A generous ceiling here is not licence to grow a scope into it -- see the + # note in stryker.hooks.config.json, which is already at its practical limit + # and wants splitting rather than more files. + timeout-minutes: 120 strategy: # A scope that lands below its floor must not cancel the other three. The # point of the weekly run is a complete picture; losing three numbers to one diff --git a/app/ui/src/hooks/useMatrix.test.jsx b/app/ui/src/hooks/useMatrix.test.jsx index e8c00add4..3f5b67788 100644 --- a/app/ui/src/hooks/useMatrix.test.jsx +++ b/app/ui/src/hooks/useMatrix.test.jsx @@ -134,3 +134,172 @@ describe('useMatrix', () => { expect(result.current.loading).toBe(false); }); }); + +// ── Response shape ────────────────────────────────────────────────────────── +// Every field of a roll-up or counts payload is read as `body.x || `. +// That single expression carries two failures and both are silent. Lose the +// default and `undefined` reaches the renderer, which draws an empty matrix that +// looks exactly like "this person has no access". Let the default win over a +// present value and real data disappears the same way. +// +// The existing roll-up test asserted two of the seventeen fields, so the other +// fifteen could have defaulted over live data with nothing failing. +describe('useMatrix response mapping', () => { + const REFERENCE = { + '/api/access-package-groups': [], + '/api/entity-tags': [], + '/api/user-columns': [], + '/api/admin/dashboard-stats': { hasData: true }, + '/api/matrix/default-filter': null, + }; + const FILTER = { conditions: [{ field: 'x', value: 'y' }] }; + + const runWith = (matrixBody) => renderHook(() => useMatrix(FILTER), { + wrapper: makeWrapper({ + auth: { authFetch: makeAuthFetch({ '/api/matrix/data': matrixBody, ...REFERENCE }) }, + }).wrapper, + }); + + it('passes every roll-up field through untouched', async () => { + // Each field carries a DIFFERENT value, so a mapper that reads the wrong + // source key produces a different object rather than coincidentally the same + // one. Booleans are true and maxDepth is 4 precisely because those are not + // the defaults — a surviving `|| false` would be invisible against false. + const body = { + rollup: 'department', + rollupKind: 'context', + rollupContextId: 'ctx-9', + focusId: 'node-7', + breadcrumb: [{ id: 'b1' }], + nodes: [{ id: 'n1' }], + rollupContent: 'roles-only', + layered: true, + layeredAttributes: true, + maxDepth: 4, + resources: [{ resourceId: 'r1' }], + groupValues: ['IT'], + groupTotals: [{ groupValue: 'IT', total: 3 }], + counts: [{ resourceId: 'r1', groupValue: 'IT', directCount: 4 }], + businessRoles: [{ roleId: 'br1' }], + roleCounts: [{ roleId: 'br1', count: 2 }], + roleRows: [{ roleId: 'br1', groupValue: 'IT' }], + cells: [{ resourceId: 'r1', groupValue: 'IT' }], + }; + const { result } = runWith(body); + + await waitFor(() => expect(result.current.rollup).not.toBe(null)); + // The whole object at once: any field defaulted over, or read from the wrong + // key, fails here rather than needing its own assertion. + expect(result.current.rollup).toEqual({ + attribute: 'department', + rollupKind: 'context', + rollupContextId: 'ctx-9', + focusId: 'node-7', + breadcrumb: [{ id: 'b1' }], + nodes: [{ id: 'n1' }], + rollupContent: 'roles-only', + layered: true, + layeredAttributes: true, + maxDepth: 4, + resources: [{ resourceId: 'r1' }], + groupValues: ['IT'], + groupTotals: [{ groupValue: 'IT', total: 3 }], + counts: [{ resourceId: 'r1', groupValue: 'IT', directCount: 4 }], + businessRoles: [{ roleId: 'br1' }], + roleCounts: [{ roleId: 'br1', count: 2 }], + roleRows: [{ roleId: 'br1', groupValue: 'IT' }], + cells: [{ resourceId: 'r1', groupValue: 'IT' }], + }); + }); + + it('fills in a documented default for every roll-up field the server omits', async () => { + // A minimal payload — only the field that selects the roll-up branch at all. + // Collections must become empty arrays rather than undefined: the renderer + // maps over them, so undefined is a crash or a blank grid, not a default. + const { result } = runWith({ rollup: 'department' }); + + await waitFor(() => expect(result.current.rollup).not.toBe(null)); + expect(result.current.rollup).toEqual({ + attribute: 'department', + rollupKind: 'attribute', + rollupContextId: null, + focusId: null, + breadcrumb: [], + nodes: [], + rollupContent: 'resources-and-roles', + layered: false, + layeredAttributes: false, + maxDepth: 1, + resources: [], + groupValues: [], + groupTotals: [], + counts: [], + businessRoles: [], + roleCounts: [], + roleRows: [], + cells: [], + }); + }); + + it('passes the five headline counts through, and zeroes the ones omitted', async () => { + // Five different non-zero values: a mapper reading the wrong key swaps two of + // them, which identical numbers would hide. + const { result } = runWith({ + rows: [], subjectCount: 3, subjectTotal: 11, resourceCount: 5, + resourceTotal: 17, assignmentCount: 23, + }); + await waitFor(() => expect(result.current.counts.subjectCount).toBe(3)); + expect(result.current.counts).toEqual({ + subjectCount: 3, subjectTotal: 11, resourceCount: 5, + resourceTotal: 17, assignmentCount: 23, + }); + + const { result: sparse } = runWith({ rows: [], subjectCount: 3 }); + await waitFor(() => expect(sparse.current.counts.subjectCount).toBe(3)); + // Zero, not undefined — these render straight into the header counters. + expect(sparse.current.counts).toEqual({ + subjectCount: 3, subjectTotal: 0, resourceCount: 0, + resourceTotal: 0, assignmentCount: 0, + }); + }); +}); + +// ── Reference-data failures ───────────────────────────────────────────────── +// Four reference fetches run on mount, each with its own catch. None was ever +// made to fail, so every fallback was unexecuted — and they are not all the same +// fallback, which is the point: one fails CLOSED (no tags) and one fails OPEN +// (assume data exists). Getting either backwards is silent. +describe('useMatrix when reference data fails', () => { + const rejectFor = (needle) => makeAuthFetch(async (url) => { + if (String(url).includes(needle)) throw new Error('network'); + if (String(url).includes('/api/admin/dashboard-stats')) return { hasData: true }; + if (String(url).includes('/api/matrix/default-filter')) return { conditions: [] }; + return []; + }); + + const run = (authFetch) => renderHook(() => useMatrix(null), { + wrapper: makeWrapper({ auth: { authFetch } }).wrapper, + }); + + it('falls back to no tags when the tag fetch fails', async () => { + const { result } = run(rejectFor('/api/entity-tags')); + // An empty Map, not null: null means "still loading" to every consumer. + await waitFor(() => expect(result.current.groupTagMap).not.toBe(null)); + expect(result.current.groupTagMap.size).toBe(0); + }); + + it('assumes data exists when the has-data check fails', async () => { + // Fails OPEN on purpose. hasData=false routes the user to an empty-state + // screen telling them to import data; a transient failure must not do that + // to a tenant whose data is fine. + const { result } = run(rejectFor('/api/admin/dashboard-stats')); + await waitFor(() => expect(result.current.hasData).toBe(true)); + }); + + it('falls back to no default filter when that fetch fails', async () => { + // null, not undefined: undefined is the "not yet loaded" sentinel the matrix + // waits on, so returning it would hang the view rather than show it unfiltered. + const { result } = run(rejectFor('/api/matrix/default-filter')); + await waitFor(() => expect(result.current.defaultFilter).toBe(null)); + }); +}); diff --git a/app/ui/stryker.hooks.config.json b/app/ui/stryker.hooks.config.json index bf9109a85..069183171 100644 --- a/app/ui/stryker.hooks.config.json +++ b/app/ui/stryker.hooks.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "The first real UI scope beyond the two-file pilot, and it is deliberately the hooks rather than the components: these four decide WHICH rows and WHICH access a reviewer is shown, and each is wrong in a way nobody can see from the screen. useEntityPage owns the search, filter, tag and pagination state behind every list page -- a wrong filter silently shows a different population, and it is the least covered of the four (80.2% line). useNestedGroupExpand walks a group to the groups it is a member of, up to MAX_NEST_LEVEL, which is how inherited access appears at all; stop one level short and access simply is not displayed. useMatrix gates whether the matrix fetches anything and debounces the filter, so a fault there shows stale or empty results that look like 'no access'. useMatrixRowOrder persists custom row order under a version key -- forget to invalidate on a version bump and the user sees an order the current sort logic no longer produces. Run: cd app/ui && npm run test:mutation:hooks. TEST INCLUDE IS WIDER THAN THE FOUR HOOKS' OWN TESTS and the width was measured, not guessed -- see vitest.stryker.hooks.config.js: the hooks' own tests alone reach 70.63% branch, the full UI suite 75.46%, and the chosen include 75.46% with identical uncovered lines. MUTATOR SET: StringLiteral and ObjectLiteral excluded, matching every API config. String mutation is expensive -- it was 20-43% of the mutants in each scope measured so far -- and that cost is paid on every weekly run forever, while the yield is uneven: on prose it is near zero (68 of 72 survivors on permissions.js), and where it does pay it usually pays once. Some strings here ARE behaviour (a localStorage key, a version suffix, a query-parameter name) and those mutants go unmeasured as a result; that is the accepted trade, not an oversight. If a specific string turns out to carry a decision worth pinning, write the assertion for it rather than re-enabling the mutator wholesale. KNOWN FLAKINESS: the UI suite is stable on its own (157 files, 1273 tests green) but produced 2 then 6 different failures across two runs under coverage instrumentation at full parallelism. If this run reports scattered timeouts rather than a consistent set, lower `concurrency` before believing the number. MEASURED 54.27% AGAINST 90.3% LINE COVERAGE -- a 36-point gap, the widest of any scope here, and useMatrix.js alone reads 97.3% line and 41.22% mutation. That 56-point spread is more than double the previous worst (effectiveAccess/engine.js at 93 -> 69). The UI was the part of this codebase we knew least about and the answer is the least reassuring. RUNTIME: 891 mutants took ~24 minutes locally, against ~4 minutes for the 1,122-mutant API matrix scope -- jsdom plus a React render per mutant is roughly an order of magnitude more expensive per mutant. That is why js-mutation.yml allows 60 minutes per scope. Adding files here costs real weekly minutes; prefer a second config over growing this one past the budget. FLOOR starts at 52, just under the first measurement, and only ratchets up.", + "_comment": "The first real UI scope beyond the two-file pilot, and it is deliberately the hooks rather than the components: these four decide WHICH rows and WHICH access a reviewer is shown, and each is wrong in a way nobody can see from the screen. useEntityPage owns the search, filter, tag and pagination state behind every list page -- a wrong filter silently shows a different population, and it is the least covered of the four (80.2% line). useNestedGroupExpand walks a group to the groups it is a member of, up to MAX_NEST_LEVEL, which is how inherited access appears at all; stop one level short and access simply is not displayed. useMatrix gates whether the matrix fetches anything and debounces the filter, so a fault there shows stale or empty results that look like 'no access'. useMatrixRowOrder persists custom row order under a version key -- forget to invalidate on a version bump and the user sees an order the current sort logic no longer produces. Run: cd app/ui && npm run test:mutation:hooks. TEST INCLUDE IS WIDER THAN THE FOUR HOOKS' OWN TESTS and the width was measured, not guessed -- see vitest.stryker.hooks.config.js: the hooks' own tests alone reach 70.63% branch, the full UI suite 75.46%, and the chosen include 75.46% with identical uncovered lines. MUTATOR SET: StringLiteral and ObjectLiteral excluded, matching every API config. String mutation is expensive -- it was 20-43% of the mutants in each scope measured so far -- and that cost is paid on every weekly run forever, while the yield is uneven: on prose it is near zero (68 of 72 survivors on permissions.js), and where it does pay it usually pays once. Some strings here ARE behaviour (a localStorage key, a version suffix, a query-parameter name) and those mutants go unmeasured as a result; that is the accepted trade, not an oversight. If a specific string turns out to carry a decision worth pinning, write the assertion for it rather than re-enabling the mutator wholesale. KNOWN FLAKINESS: the UI suite is stable on its own (157 files, 1273 tests green) but produced 2 then 6 different failures across two runs under coverage instrumentation at full parallelism. If this run reports scattered timeouts rather than a consistent set, lower `concurrency` before believing the number. MEASURED 54.27% AGAINST 90.3% LINE COVERAGE -- a 36-point gap, the widest of any scope here, and useMatrix.js alone reads 97.3% line and 41.22% mutation. That 56-point spread is more than double the previous worst (effectiveAccess/engine.js at 93 -> 69). The UI was the part of this codebase we knew least about and the answer is the least reassuring. RUNTIME IS THIS SCOPE'S BINDING CONSTRAINT, AND IT GREW WITH THE TESTS. 891 mutants took 24 minutes at the first measurement and 58 minutes after six tests were added -- against ~4 minutes for the 1,122-mutant API matrix scope. That is not a regression: a mutant costs one run of every test that covers it, so better-covered code is slower to mutate, and jsdom plus a React render makes each of those runs an order of magnitude dearer than an API one. THIS SCOPE IS AT ITS PRACTICAL LIMIT. Do not add files to it. The next hook belongs in a second config, split along mount-test lines so each config's mutants run fewer of them -- useEntityPage needs only EntityListPage.mount, the matrix hooks only MatrixView.mount and App.mount, so the split cuts per-mutant cost rather than just dividing it. FLOOR ratchets up only: 52 at the first measurement, 63 once useMatrix.js went 41.22 -> 69.18 and the scope reached 65.18.", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -28,7 +28,7 @@ "thresholds": { "high": 90, "low": 70, - "break": 52 + "break": 63 }, "concurrency": 4, "timeoutMS": 60000, From d3813d337df5506fde5124fe5918f1a875681e0b Mon Sep 17 00:00:00 2001 From: Taeke Date: Wed, 19 Aug 2026 08:59:27 +0200 Subject: [PATCH 13/15] ci: split the UI hook scope, and cut its run from 58 minutes to 14 The UI hooks scope was measured at 58 minutes against a 60-minute job budget. Raising the ceiling was the wrong fix and the config said so; this is the right one. The cost was never the mutant count. A mutant costs one run of EVERY test in its include set, and useEntityPage's ~400 mutants were each loading MatrixView.mount and App.mount for nothing, while the matrix hooks' mutants each loaded EntityListPage.mount for nothing. Splitting on which mount test a hook actually needs makes each mutant cheaper, not just fewer per job: before 1 config 4 files 891 mutants 58m12s after hooks 3 files 489 mutants 10m57s 70.07% listhooks 1 file 402 mutants 13m58s 57.50% Total CPU 25 minutes against 58, and the two run as parallel matrix entries, so wall-clock is ~14. The job timeout goes back to 60: nothing is near it now, and a scope that approaches it should be split rather than given more time. Floors are each measured over the scope that now exists -- hooks 68, listhooks 55. The combined 65.18 does not carry over to either; the denominator changed. PARITY WAS CHECKED FOR BOTH HALVES, NOT ASSUMED. Half A matched the full UI suite immediately. Half B was 4 branches short, and diffing the branch maps named them exactly: the failure arms of useEntityPage's columns and tags fetches, reached only incidentally by a mount test elsewhere in the suite. That is both the wrong home for the contract and precisely the dependency that becomes a false survivor when a scope narrows, so the tests were written where the behaviour lives. What must not happen is the point of them: a failed columns fetch has to leave the previous columns alone, because blanking them removes every filter the user could apply and an empty filter bar reads as "nothing to filter on". Two harness traps, both of which cost a red test first and are worth knowing: makeAuthFetch matches URLs by substring, so listing '/api/users' before '/api/users/columns' answers the columns request with the list payload -- an object where an array is expected, failing deep inside getFilterFields rather than at the mock. And getFilterFields dereferences the `fieldLabels` prop with no default, so it throws for any caller that omits one; asserted around it here rather than changing production code mid-task, but it is a real latent crash. One number in this file's history is a lie worth documenting: an overnight run of listhooks reported 585 minutes. The machine slept. One full pass of its two test files takes 5.5 seconds and re-running it awake gave 14 minutes. The caveat is in the config so the next person checks the host before tuning anything. --- .github/workflows/js-mutation.yml | 25 ++++---- app/ui/package.json | 5 +- app/ui/src/hooks/useEntityPage.test.jsx | 74 +++++++++++++++++++++++ app/ui/stryker.hooks.config.json | 5 +- app/ui/stryker.listhooks.config.json | 33 ++++++++++ app/ui/vitest.stryker.hooks.config.js | 18 +++--- app/ui/vitest.stryker.listhooks.config.js | 37 ++++++++++++ 7 files changed, 170 insertions(+), 27 deletions(-) create mode 100644 app/ui/stryker.listhooks.config.json create mode 100644 app/ui/vitest.stryker.listhooks.config.js diff --git a/.github/workflows/js-mutation.yml b/.github/workflows/js-mutation.yml index 08dd15b56..8344a0b1a 100644 --- a/.github/workflows/js-mutation.yml +++ b/.github/workflows/js-mutation.yml @@ -59,17 +59,17 @@ jobs: mutation: name: 'Mutation: ${{ matrix.scope.label }}' runs-on: ubuntu-latest - # 120, because the UI scopes are an order of magnitude more expensive per - # mutant than the API ones: every mutant re-renders React under jsdom. - # Measured locally -- the 891-mutant UI hooks scope took 24 minutes when first - # opened and 58 after six tests were added, against ~4 minutes for 1,122 API - # mutants. A mutant costs one run of every test covering it, so the better the - # tests get the slower the run: the budget has to hold the scope's FUTURE cost, - # not today's, and CI hardware is slower than the machine that measured this. - # A generous ceiling here is not licence to grow a scope into it -- see the - # note in stryker.hooks.config.json, which is already at its practical limit - # and wants splitting rather than more files. - timeout-minutes: 120 + # 60. The UI scopes are an order of magnitude more expensive per mutant than + # the API ones -- every mutant re-renders React under jsdom -- but no scope is + # near this now: measured locally, the slowest is 14 minutes (UI list hook, + # 402 mutants) against ~5 for the 1,122-mutant API matrix scope. This was + # briefly 120, when the UI hooks lived in one config and took 58 minutes; the + # fix was splitting that config rather than buying it more time, because a + # mutant costs one run of EVERY test in its include set. Keep the margin for + # slower CI hardware and for tests yet to be written -- better-covered code is + # slower to mutate -- but treat a scope approaching this as a signal to split + # it, not to raise the number. + timeout-minutes: 60 strategy: # A scope that lands below its floor must not cancel the other three. The # point of the weekly run is a complete picture; losing three numbers to one @@ -82,7 +82,8 @@ jobs: - { pkg: api, name: accountlinking, label: 'API account linking' } - { pkg: api, name: matrix, label: 'API matrix' } - { pkg: ui, name: pilot, label: 'UI permissions + filter' } - - { pkg: ui, name: hooks, label: 'UI matrix + list hooks' } + - { pkg: ui, name: hooks, label: 'UI matrix hooks' } + - { pkg: ui, name: listhooks, label: 'UI list-page hook' } steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: diff --git a/app/ui/package.json b/app/ui/package.json index 9a0a61eb6..55eb95c10 100644 --- a/app/ui/package.json +++ b/app/ui/package.json @@ -17,9 +17,10 @@ "test:e2e:sql": "bash scripts/e2e-sql.sh", "test:e2e:headed": "npx playwright test --headed", "test:e2e:ui": "npx playwright test --ui", - "test:mutation": "npm run test:mutation:pilot && npm run test:mutation:hooks", + "test:mutation": "npm run test:mutation:pilot && npm run test:mutation:hooks && npm run test:mutation:listhooks", "test:mutation:pilot": "stryker run stryker.pilot.config.json", - "test:mutation:hooks": "stryker run stryker.hooks.config.json" + "test:mutation:hooks": "stryker run stryker.hooks.config.json", + "test:mutation:listhooks": "stryker run stryker.listhooks.config.json" }, "dependencies": { "@azure/msal-browser": "^5.18.0", diff --git a/app/ui/src/hooks/useEntityPage.test.jsx b/app/ui/src/hooks/useEntityPage.test.jsx index 7a303e670..aa13bd762 100644 --- a/app/ui/src/hooks/useEntityPage.test.jsx +++ b/app/ui/src/hooks/useEntityPage.test.jsx @@ -407,3 +407,77 @@ describe('useEntityPage', () => { expect(groups.result.current.search).toBe(''); }); }); + +// ── When the sidecar fetches fail ─────────────────────────────────────────── +// Columns and tags are loaded alongside the list, and each has the same two-step +// guard: `res.ok ? res.json() : null`, then `if (data) setX(data)`. Neither +// failure arm was reachable from this file -- only a mount test elsewhere in the +// suite happened to drive them, which is a poor place for the contract to live +// and left these mutants unkilled here. +// +// What must NOT happen is the interesting part: a failed columns fetch must leave +// the previous columns alone rather than blanking them, because the filter bar is +// built from that list. Overwriting it with null or [] silently removes every +// filter the user could apply, and an empty filter bar looks like a page with +// nothing to filter on rather than a page whose request failed. +describe('useEntityPage sidecar failures', () => { + // KEY ORDER MATTERS: makeAuthFetch matches by substring, and the columns URL + // ('/api/users/columns') contains the list URL ('/api/users'). List-first would + // answer the columns request with the list payload -- an object where an array + // is expected, which fails deep inside getFilterFields rather than at the mock. + const listOnly = (extra = {}) => ({ + ...extra, + [LIST]: { data: [{ id: '1', displayName: 'Bob' }], total: 1 }, + }); + + it('keeps the list usable when the columns request fails', async () => { + const { result } = setup({ + handler: listOnly({ + [COLUMNS]: jsonResponse({ error: 'nope' }, { ok: false, status: 500 }), + '/api/tags': [{ id: 't1', name: 'VIP' }], + }), + }); + + await waitFor(() => expect(result.current.loading).toBe(false)); + // The rows still arrive — one failed sidecar does not take the page down. + expect(result.current.items).toEqual([{ id: '1', displayName: 'Bob' }]); + // ...and the loading flag still settles, or the filter bar spins forever. + await waitFor(() => expect(result.current.columnsLoading).toBe(false)); + // Tags are unaffected: the two fetches fail independently. + expect(result.current.tags).toEqual([{ id: 't1', name: 'VIP' }]); + }); + + it('keeps the list usable when the tags request fails', async () => { + const { result } = setup({ + handler: listOnly({ + [COLUMNS]: [{ column: 'department', values: ['Sales'] }], + '/api/tags': jsonResponse({ error: 'nope' }, { ok: false, status: 503 }), + }), + }); + + await waitFor(() => expect(result.current.loading).toBe(false)); + expect(result.current.items).toEqual([{ id: '1', displayName: 'Bob' }]); + // Columns are unaffected — again, independent failures. Asserted through + // getOptionsForField rather than getFilterFields: the latter dereferences the + // `fieldLabels` prop with no default, so it throws for any caller that omits + // one. Worth knowing, but not this test's subject. + await waitFor(() => expect(result.current.columnsLoading).toBe(false)); + expect(result.current.getOptionsForField('department')).toEqual(['Sales']); + }); + + it('leaves both sidecars at their defaults when both fail', async () => { + const { result } = setup({ + handler: listOnly({ + [COLUMNS]: jsonResponse({ error: 'nope' }, { ok: false, status: 500 }), + '/api/tags': jsonResponse({ error: 'nope' }, { ok: false, status: 500 }), + }), + }); + + await waitFor(() => expect(result.current.loading).toBe(false)); + await waitFor(() => expect(result.current.columnsLoading).toBe(false)); + // Both stay empty rather than becoming null: every consumer maps over them. + expect(result.current.tags).toEqual([]); + expect(result.current.getOptionsForField('department')).toEqual([]); + expect(result.current.items).toEqual([{ id: '1', displayName: 'Bob' }]); + }); +}); diff --git a/app/ui/stryker.hooks.config.json b/app/ui/stryker.hooks.config.json index 069183171..b0c4abe2d 100644 --- a/app/ui/stryker.hooks.config.json +++ b/app/ui/stryker.hooks.config.json @@ -1,6 +1,6 @@ { "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", - "_comment": "The first real UI scope beyond the two-file pilot, and it is deliberately the hooks rather than the components: these four decide WHICH rows and WHICH access a reviewer is shown, and each is wrong in a way nobody can see from the screen. useEntityPage owns the search, filter, tag and pagination state behind every list page -- a wrong filter silently shows a different population, and it is the least covered of the four (80.2% line). useNestedGroupExpand walks a group to the groups it is a member of, up to MAX_NEST_LEVEL, which is how inherited access appears at all; stop one level short and access simply is not displayed. useMatrix gates whether the matrix fetches anything and debounces the filter, so a fault there shows stale or empty results that look like 'no access'. useMatrixRowOrder persists custom row order under a version key -- forget to invalidate on a version bump and the user sees an order the current sort logic no longer produces. Run: cd app/ui && npm run test:mutation:hooks. TEST INCLUDE IS WIDER THAN THE FOUR HOOKS' OWN TESTS and the width was measured, not guessed -- see vitest.stryker.hooks.config.js: the hooks' own tests alone reach 70.63% branch, the full UI suite 75.46%, and the chosen include 75.46% with identical uncovered lines. MUTATOR SET: StringLiteral and ObjectLiteral excluded, matching every API config. String mutation is expensive -- it was 20-43% of the mutants in each scope measured so far -- and that cost is paid on every weekly run forever, while the yield is uneven: on prose it is near zero (68 of 72 survivors on permissions.js), and where it does pay it usually pays once. Some strings here ARE behaviour (a localStorage key, a version suffix, a query-parameter name) and those mutants go unmeasured as a result; that is the accepted trade, not an oversight. If a specific string turns out to carry a decision worth pinning, write the assertion for it rather than re-enabling the mutator wholesale. KNOWN FLAKINESS: the UI suite is stable on its own (157 files, 1273 tests green) but produced 2 then 6 different failures across two runs under coverage instrumentation at full parallelism. If this run reports scattered timeouts rather than a consistent set, lower `concurrency` before believing the number. MEASURED 54.27% AGAINST 90.3% LINE COVERAGE -- a 36-point gap, the widest of any scope here, and useMatrix.js alone reads 97.3% line and 41.22% mutation. That 56-point spread is more than double the previous worst (effectiveAccess/engine.js at 93 -> 69). The UI was the part of this codebase we knew least about and the answer is the least reassuring. RUNTIME IS THIS SCOPE'S BINDING CONSTRAINT, AND IT GREW WITH THE TESTS. 891 mutants took 24 minutes at the first measurement and 58 minutes after six tests were added -- against ~4 minutes for the 1,122-mutant API matrix scope. That is not a regression: a mutant costs one run of every test that covers it, so better-covered code is slower to mutate, and jsdom plus a React render makes each of those runs an order of magnitude dearer than an API one. THIS SCOPE IS AT ITS PRACTICAL LIMIT. Do not add files to it. The next hook belongs in a second config, split along mount-test lines so each config's mutants run fewer of them -- useEntityPage needs only EntityListPage.mount, the matrix hooks only MatrixView.mount and App.mount, so the split cuts per-mutant cost rather than just dividing it. FLOOR ratchets up only: 52 at the first measurement, 63 once useMatrix.js went 41.22 -> 69.18 and the scope reached 65.18.", + "_comment": "The first real UI scope beyond the two-file pilot, and it is deliberately the hooks rather than the components: these three decide WHICH rows and WHICH access a reviewer is shown, and each is wrong in a way nobody can see from the screen. useNestedGroupExpand walks a group to the groups it is a member of, up to MAX_NEST_LEVEL, which is how inherited access appears at all; stop one level short and access simply is not displayed. useMatrix gates whether the matrix fetches anything and debounces the filter, so a fault there shows stale or empty results that look like 'no access'. useMatrixRowOrder persists custom row order under a version key -- forget to invalidate on a version bump and the user sees an order the current sort logic no longer produces. Run: cd app/ui && npm run test:mutation:hooks. TEST INCLUDE IS WIDER THAN THE THREE HOOKS' OWN TESTS and the width was measured, not guessed -- see vitest.stryker.hooks.config.js: the hooks' own tests alone reach 70.63% branch, the full UI suite 75.46%, and the chosen include 75.46% with identical uncovered lines. MUTATOR SET: StringLiteral and ObjectLiteral excluded, matching every API config. String mutation is expensive -- it was 20-43% of the mutants in each scope measured so far -- and that cost is paid on every weekly run forever, while the yield is uneven: on prose it is near zero (68 of 72 survivors on permissions.js), and where it does pay it usually pays once. Some strings here ARE behaviour (a localStorage key, a version suffix, a query-parameter name) and those mutants go unmeasured as a result; that is the accepted trade, not an oversight. If a specific string turns out to carry a decision worth pinning, write the assertion for it rather than re-enabling the mutator wholesale. KNOWN FLAKINESS: the UI suite is stable on its own (157 files, 1273 tests green) but produced 2 then 6 different failures across two runs under coverage instrumentation at full parallelism. If this run reports scattered timeouts rather than a consistent set, lower `concurrency` before believing the number. MEASURED 54.27% AGAINST 90.3% LINE COVERAGE -- a 36-point gap, the widest of any scope here, and useMatrix.js alone reads 97.3% line and 41.22% mutation. That 56-point spread is more than double the previous worst (effectiveAccess/engine.js at 93 -> 69). The UI was the part of this codebase we knew least about and the answer is the least reassuring. RUNTIME: 489 mutants in 11 minutes. It was 58 minutes before the split, for four files and 891 mutants -- and the split is why it fell, not the smaller mutant count alone: a mutant costs one run of EVERY test in the include set, so dropping EntityListPage.mount from this config made each of these 489 mutants cheaper as well as fewer. Watch this number when adding tests: better-covered code is slower to mutate, and a React render under jsdom is an order of magnitude dearer per run than an API test. THE SPLIT HAPPENED: useEntityPage moved to stryker.listhooks.config.json, because it needs only EntityListPage.mount while these three need MatrixView.mount and App.mount. Each config's mutants now load fewer tests, which cuts per-mutant cost rather than merely dividing the same work between two jobs. Keep that seam when adding a hook: put it where its mount test already is, or give it its own config. FLOOR 68, from the narrowed scope's own measurement of 70.07. The combined four-file scope scored 65.18 and that number does not carry over -- the denominator changed when useEntityPage left. Ratchets up only.", "packageManager": "npm", "testRunner": "vitest", "vitest": { @@ -14,7 +14,6 @@ "fileName": "reports/stryker-ui-hooks.json" }, "mutate": [ - "src/hooks/useEntityPage.js", "src/hooks/useMatrix.js", "src/hooks/useMatrixRowOrder.js", "src/hooks/useNestedGroupExpand.js" @@ -28,7 +27,7 @@ "thresholds": { "high": 90, "low": 70, - "break": 63 + "break": 68 }, "concurrency": 4, "timeoutMS": 60000, diff --git a/app/ui/stryker.listhooks.config.json b/app/ui/stryker.listhooks.config.json new file mode 100644 index 000000000..c4d57d717 --- /dev/null +++ b/app/ui/stryker.listhooks.config.json @@ -0,0 +1,33 @@ +{ + "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", + "_comment": "useEntityPage — the search, filter, tag and pagination state behind every list page (Users, Groups, Identities). A wrong filter here silently shows a different population, and nothing on screen says the list is not what was asked for. Split out of stryker.hooks.config.json rather than added to it: measured together the two scopes took 58 minutes against a 60-minute CI budget, because every mutant pays for every test in the include set, and useEntityPage's ~320 mutants were each loading MatrixView.mount and App.mount for nothing. Splitting on which mount test each hook actually needs cuts per-mutant cost instead of dividing the same work. Run: cd app/ui && npm run test:mutation:listhooks. TEST INCLUDE is useEntityPage.test.jsx plus EntityListPage.mount.test.jsx, and parity was checked rather than assumed -- coverage under those two matches the full UI suite exactly (79.69/71.71/84.31/80.24, same uncovered lines). It was 4 branches short until the sidecar-failure tests were written: the failure arms of the columns and tags fetches had only ever been reached incidentally by a mount test elsewhere in the suite, which is both the wrong home for that contract and precisely the dependency that becomes a false survivor when a scope narrows. MUTATOR SET: StringLiteral and ObjectLiteral excluded, matching every other config here -- string mutation was 20-43% of the mutants in each scope measured and is re-paid every week for an uneven return. MEASURED 57.50% over 402 mutants in 14 minutes; FLOOR 55, ratcheting up only. 64 of its mutants are unreached, so there is a lot left here -- the file is 162 lines of search, filter, tag and pagination state and the least covered of the UI hooks at 80.2% line. RUNTIME CAVEAT WORTH KEEPING: an overnight run of this same config reported 585 minutes. That was the machine sleeping, not the work -- one full pass of its two test files takes 5.5 seconds, and re-running it awake gave 14 minutes. If a mutation run ever reports hours, check whether the host stayed awake before believing it or tuning anything.", + "packageManager": "npm", + "testRunner": "vitest", + "vitest": { + "configFile": "vitest.stryker.listhooks.config.js" + }, + "reporters": [ + "clear-text", + "json" + ], + "jsonReporter": { + "fileName": "reports/stryker-ui-listhooks.json" + }, + "mutate": [ + "src/hooks/useEntityPage.js" + ], + "mutator": { + "excludedMutations": [ + "StringLiteral", + "ObjectLiteral" + ] + }, + "thresholds": { + "high": 90, + "low": 70, + "break": 55 + }, + "concurrency": 4, + "timeoutMS": 60000, + "disableTypeChecks": false +} diff --git a/app/ui/vitest.stryker.hooks.config.js b/app/ui/vitest.stryker.hooks.config.js index 74bb25ede..01d1f3195 100644 --- a/app/ui/vitest.stryker.hooks.config.js +++ b/app/ui/vitest.stryker.hooks.config.js @@ -10,16 +10,14 @@ import path from 'path'; // run before a mutant is evaluated, and the tailwind plugin plus the '@crawlers' // alias are build concerns that buy the mutation run nothing. // -// WIDER THAN THE FOUR HOOKS' OWN TESTS, AND MEASURED RATHER THAN GUESSED. The -// obvious include — the four *.test.jsx files next to the hooks — leaves branches +// WIDER THAN THE THREE HOOKS' OWN TESTS, AND MEASURED RATHER THAN GUESSED. The +// obvious include — just the *.test.jsx files next to the hooks — leaves branches // unexercised that the component mount tests do reach, and every one of those // would have come back a SURVIVOR that no test could ever kill. Measured before -// choosing: coverage of the four hooks under the hooks' own tests alone is -// 70.63% branch, under the full UI suite 75.46%, and under the list below 75.46% -// — identical to the full suite, with the same uncovered lines. So the narrowing -// costs nothing. +// choosing: coverage of the three hooks under the list below is 94.33% stmts / +// 82.94% branch / 87.83% funcs / 99.52% lines — identical to the full UI suite, +// with the same uncovered lines. So the narrowing costs nothing. // -// EntityListPage.mount.test.jsx drives useEntityPage through the real list page // MatrixView.mount.test.jsx drives useMatrixRowOrder + useNestedGroupExpand // App.mount.test.jsx the only test that mounts the useMatrix caller // nestedRows.helpers.test.js the row builder that consumes the expand cache @@ -28,7 +26,9 @@ import path from 'path'; // of these numbers, so it cannot be any mutant's only killer here. // // Re-measure this comparison when adding a hook to the `mutate` list — the answer -// is per-file, not a property of the directory. +// is per-file, not a property of the directory. And prefer a new config over a +// longer include: every mutant pays for every test listed here, which is why +// useEntityPage moved out to vitest.stryker.listhooks.config.js. export default defineConfig({ plugins: [react()], @@ -39,9 +39,7 @@ export default defineConfig({ include: [ 'src/hooks/useMatrix.test.jsx', 'src/hooks/useNestedGroupExpand.test.jsx', - 'src/hooks/useEntityPage.test.jsx', 'src/hooks/useMatrixRowOrder.test.js', - 'src/components/EntityListPage.mount.test.jsx', 'src/components/MatrixView.mount.test.jsx', 'src/App.mount.test.jsx', 'src/components/matrix/nestedRows.helpers.test.js', diff --git a/app/ui/vitest.stryker.listhooks.config.js b/app/ui/vitest.stryker.listhooks.config.js new file mode 100644 index 000000000..7f39a0e08 --- /dev/null +++ b/app/ui/vitest.stryker.listhooks.config.js @@ -0,0 +1,37 @@ +import { defineConfig } from 'vite'; +import react from '@vitejs/plugin-react'; +import path from 'path'; + +// Vitest config for the list-page hook mutation run (stryker.listhooks.config.json). +// +// Standalone rather than spreading vite.config.js, for the same reason as the other +// vitest.stryker.* configs: Stryker sandboxes app/ui alone, so the normal test include +// reaching ../../tools/crawlers/** resolves nothing there and aborts the run. +// +// TWO TEST FILES, AND THAT IS THE POINT OF THIS CONFIG EXISTING. useEntityPage was +// originally measured alongside the matrix hooks, which meant every one of its ~320 +// mutants also paid for MatrixView.mount and App.mount to load. Splitting on which +// mount test each hook actually needs cuts per-mutant cost rather than merely dividing +// the same work between two jobs. +// +// Parity checked, not assumed: coverage of useEntityPage under these two files is +// 79.69% stmts / 71.71% branch / 84.31% funcs / 80.24% lines, identical to the full UI +// suite with the same uncovered lines. It was 4 branches short until the sidecar-failure +// tests were added to useEntityPage.test.jsx — those arms had only ever been reached +// incidentally by a mount test elsewhere in the suite, which is both a poor place for +// the contract to live and exactly the kind of dependency that turns into a false +// survivor when a scope is narrowed. + +export default defineConfig({ + plugins: [react()], + resolve: { + alias: { '@ui': path.resolve(import.meta.dirname, 'src') }, + }, + test: { + include: [ + 'src/hooks/useEntityPage.test.jsx', + 'src/components/EntityListPage.mount.test.jsx', + ], + exclude: ['**/node_modules/**'], + }, +}); From dcca3099dd835460d229cefeea5fb80f246311cd Mon Sep 17 00:00:00 2001 From: Taeke Date: Wed, 19 Aug 2026 12:17:16 +0200 Subject: [PATCH 14/15] docs: give this PR its own changelog fragment Same reason as the parent branches: bump-version.yml merges every changes/*.md into CHANGES.md and then deletes them, so a stack sharing one cumulative fragment republishes the lower PRs' bullets each time a higher one merges. This branch's four bullets move to a file named after it. --- changes/ui-mutation-scope.md | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 changes/ui-mutation-scope.md diff --git a/changes/ui-mutation-scope.md b/changes/ui-mutation-scope.md new file mode 100644 index 000000000..e6cdddd07 --- /dev/null +++ b/changes/ui-mutation-scope.md @@ -0,0 +1,4 @@ +- Started measuring the web interface properly, beginning with the four pieces of logic that decide which rows and which access a reviewer is shown: the search/filter/pagination behind every list page, the walk that expands a group into the groups it belongs to (which is how inherited access appears at all), the matrix's data loading, and the saved row order. Only 54% of injected faults are caught, against a line coverage of 90% — the widest gap found anywhere so far. The matrix data hook is the starkest: 97% of its lines run under test, and fewer than half of the faults planted in it are noticed. +- Stopped mutating text literals in the checks that were doing so. It was a quarter to nearly half of the work in every area measured, it is repeated every week, and on descriptive text it finds essentially nothing. The tests written to catch those faults remain; only the repeated re-checking of them stops. +- Closed the worst of the gaps found in the matrix's data loading. The seventeen fields of a matrix response are each read with a fallback for when the server omits them, and only two were ever checked — so fifteen could have been quietly replacing live data with an empty default, which on screen is indistinguishable from "this person has no access". The four pieces of reference data loaded when the matrix opens each have their own fallback for a failed request, and none had ever been made to fail; they are not all the same fallback, and one deliberately assumes data exists so that a momentary network problem cannot send someone to a "no data — import some" screen when their data is fine. +- Made the list pages' search and filter behaviour survive a failed request. The lists of columns and tags a page offers to filter by are loaded alongside the rows, and neither failure path had ever been exercised; a failed load must leave the previous choices in place, because blanking them removes every filter the user could apply and an empty filter bar reads as "nothing to filter on" rather than "that request failed". From 4345d1b150495e41d4cce413997a97f998070b66 Mon Sep 17 00:00:00 2001 From: Taeke Date: Thu, 20 Aug 2026 13:22:26 +0200 Subject: [PATCH 15/15] test: drive both roll-up assertions from one field table jscpd's PR-delta gate caught a clone this PR introduced: the two roll-up tests were 18-line object literals differing only in values, base 83 clones -> head 84. The aggregate stayed at 0.68% and would never have shown it, which is exactly why that second check exists. Each field is now one entry pairing what the server sent with what it defaults to when omitted, and both tests derive from it. Better than the duplicate it replaces: the two lists could previously drift the moment a field was added to one and not the other, and the correspondence between a value and its default was implicit in matching line order. Added a test that guards the fixture rather than the hook: every populated value must differ from its own default. If one ever equals it, the passthrough test still passes while proving nothing about that field -- a mutant swapping them would be undetectable. That property was previously only a comment. --- app/ui/src/hooks/useMatrix.test.jsx | 122 ++++++++++++---------------- 1 file changed, 52 insertions(+), 70 deletions(-) diff --git a/app/ui/src/hooks/useMatrix.test.jsx b/app/ui/src/hooks/useMatrix.test.jsx index 3f5b67788..9b89155b7 100644 --- a/app/ui/src/hooks/useMatrix.test.jsx +++ b/app/ui/src/hooks/useMatrix.test.jsx @@ -160,85 +160,67 @@ describe('useMatrix response mapping', () => { }).wrapper, }); - it('passes every roll-up field through untouched', async () => { - // Each field carries a DIFFERENT value, so a mapper that reads the wrong - // source key produces a different object rather than coincidentally the same - // one. Booleans are true and maxDepth is 4 precisely because those are not - // the defaults — a surviving `|| false` would be invisible against false. - const body = { - rollup: 'department', - rollupKind: 'context', - rollupContextId: 'ctx-9', - focusId: 'node-7', - breadcrumb: [{ id: 'b1' }], - nodes: [{ id: 'n1' }], - rollupContent: 'roles-only', - layered: true, - layeredAttributes: true, - maxDepth: 4, - resources: [{ resourceId: 'r1' }], - groupValues: ['IT'], - groupTotals: [{ groupValue: 'IT', total: 3 }], - counts: [{ resourceId: 'r1', groupValue: 'IT', directCount: 4 }], - businessRoles: [{ roleId: 'br1' }], - roleCounts: [{ roleId: 'br1', count: 2 }], - roleRows: [{ roleId: 'br1', groupValue: 'IT' }], - cells: [{ resourceId: 'r1', groupValue: 'IT' }], - }; - const { result } = runWith(body); + // Each roll-up field paired with (what the server sent, what it defaults to when + // the server omits it). One table drives both tests below: previously they were + // two 18-line object literals differing only in values, which is a clone by any + // measure and drifts the moment a field is added to one and not the other. + // + // Every populated value is DIFFERENT from its default, and deliberately so -- + // booleans are true, maxDepth is 4 -- because a surviving `|| false` is invisible + // against a fixture that also says false. The test below asserts that property of + // the table itself, so the fixture cannot quietly stop discriminating. + const ROLLUP_FIELDS = { + rollupKind: ['context', 'attribute'], + rollupContextId: ['ctx-9', null], + focusId: ['node-7', null], + breadcrumb: [[{ id: 'b1' }], []], + nodes: [[{ id: 'n1' }], []], + rollupContent: ['roles-only', 'resources-and-roles'], + layered: [true, false], + layeredAttributes: [true, false], + maxDepth: [4, 1], + resources: [[{ resourceId: 'r1' }], []], + groupValues: [['IT'], []], + groupTotals: [[{ groupValue: 'IT', total: 3 }], []], + counts: [[{ resourceId: 'r1', groupValue: 'IT', directCount: 4 }], []], + businessRoles: [[{ roleId: 'br1' }], []], + roleCounts: [[{ roleId: 'br1', count: 2 }], []], + roleRows: [[{ roleId: 'br1', groupValue: 'IT' }], []], + cells: [[{ resourceId: 'r1', groupValue: 'IT' }], []], + }; + const pick = (i) => Object.fromEntries( + Object.entries(ROLLUP_FIELDS).map(([k, pair]) => [k, pair[i]])); + const SENT = pick(0); + const DEFAULTS = pick(1); - await waitFor(() => expect(result.current.rollup).not.toBe(null)); - // The whole object at once: any field defaulted over, or read from the wrong - // key, fails here rather than needing its own assertion. - expect(result.current.rollup).toEqual({ - attribute: 'department', - rollupKind: 'context', - rollupContextId: 'ctx-9', - focusId: 'node-7', - breadcrumb: [{ id: 'b1' }], - nodes: [{ id: 'n1' }], - rollupContent: 'roles-only', - layered: true, - layeredAttributes: true, - maxDepth: 4, - resources: [{ resourceId: 'r1' }], - groupValues: ['IT'], - groupTotals: [{ groupValue: 'IT', total: 3 }], - counts: [{ resourceId: 'r1', groupValue: 'IT', directCount: 4 }], - businessRoles: [{ roleId: 'br1' }], - roleCounts: [{ roleId: 'br1', count: 2 }], - roleRows: [{ roleId: 'br1', groupValue: 'IT' }], - cells: [{ resourceId: 'r1', groupValue: 'IT' }], + it('uses a fixture where no field can pass by coincidence', () => { + // Guards the table, not the hook. If a populated value ever equals its own + // default, the passthrough test below still passes while proving nothing about + // that field -- the mutant that swaps them would be undetectable. + const indistinguishable = Object.entries(ROLLUP_FIELDS) + .filter(([, [sent, dflt]]) => JSON.stringify(sent) === JSON.stringify(dflt)) + .map(([k]) => k); + expect(indistinguishable).toEqual([]); + }); + + it('passes every roll-up field through untouched', () => { + // `attribute` is the one field read from a differently-named key (body.rollup), + // so it sits outside the table. + const { result } = runWith({ rollup: 'department', ...SENT }); + + return waitFor(() => expect(result.current.rollup).not.toBe(null)).then(() => { + expect(result.current.rollup).toEqual({ attribute: 'department', ...SENT }); }); }); it('fills in a documented default for every roll-up field the server omits', async () => { // A minimal payload — only the field that selects the roll-up branch at all. - // Collections must become empty arrays rather than undefined: the renderer - // maps over them, so undefined is a crash or a blank grid, not a default. + // Collections must become empty arrays rather than undefined: the renderer maps + // over them, so undefined is a crash or a blank grid, not a default. const { result } = runWith({ rollup: 'department' }); await waitFor(() => expect(result.current.rollup).not.toBe(null)); - expect(result.current.rollup).toEqual({ - attribute: 'department', - rollupKind: 'attribute', - rollupContextId: null, - focusId: null, - breadcrumb: [], - nodes: [], - rollupContent: 'resources-and-roles', - layered: false, - layeredAttributes: false, - maxDepth: 1, - resources: [], - groupValues: [], - groupTotals: [], - counts: [], - businessRoles: [], - roleCounts: [], - roleRows: [], - cells: [], - }); + expect(result.current.rollup).toEqual({ attribute: 'department', ...DEFAULTS }); }); it('passes the five headline counts through, and zeroes the ones omitted', async () => {