Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Contribution Path & Summary
CORE_CHANGE_PATHExpose the existing offline Semantic Core aggregation as
semanticCoreMetricsForand isolate returned metric identities and benchmark bindings, including nested benchmark identities, from shared canonical objects. Previously, mutation through a returned result could affect later results or canonical definitions. Each result now receives independent identity objects.Type:
fix/refactor. Domain: Benchmark / Security / Core API. Branch:refactor/semantic-core-metric-aggregation.Core classification applies because the change repairs a canonical-identity trust boundary and intentionally changes object-reference behavior. Serialized metrics, formulas, eligibility, missingness, numerators, denominators, metric versions, evaluator identity and provenance-reference format remain unchanged.
Related historical scope: Issue #165 and Draft PR #166. API contract: Semantic Core pilot guide. These references do not establish human approval of this repair. Maintainers must confirm the applicable Core/RFC process; no accepted repair-specific RFC is asserted.
2. Tests & Verification
Current synchronized head:
cf920d66873cff992daab0314881f9a4795eb664. API repair head before synchronization:0be6214ff0fdb54c757392f13f5ff1b18978d1dd.Current exact-head GitHub CI: PENDING — 8 required contexts passed, 0 failed, 1 pending; observed 2026-10-08T22:12:43.068Z.
Documentation Site: completed / success.
Security: completed / success.
CI: in_progress / pending.
Earlier checks on the pre-synchronization repair head were all SUCCESS. They are historical evidence and are not reused as CI for the new head.
Post-security isolated integration reviewed against main
24c885c0cfc46bd3eb35fc38189397683b47ff59. Conflict-free combined Git tree:0fb29a78788dd2ec4473669b22365f30152d087e; the synchronized commit has exactly this tree. Current protected main was integrated with a normal non-force merge, normal repository hooks, and one branch push; no merge into main occurred.Combined-source local results:
pnpm test: 1,526 passed, 36 skipped, 0 failed; 238 files passed, 10 skipped. Skips remain skips; this is not a claim that every test executed.Local environment: Windows, Node 24.20.0, pnpm 11.11.0. CI uses Linux/Node 22. The local installation used the frozen lockfile, offline cache and
--ignore-scripts; it is not an identical CI environment. Python build retries followed installation of the declared Hatchling backend in the isolated environments; initial missing-backend failures are retained in the operational report. No source fix was needed.3. Compatibility & Cross-Language Parity
Existing five S03 metrics and serialized results remain compatible. No new schema, metric, benchmark version, Python API, evaluator or transport was added. The benchmark package remains private and the helper remains an internal experimental API.
Trusted-input contract: callers supply already evaluated case records and attest
packDigest. Aggregation does not independently validate schemas, identities, uniqueness, digest correctness or provenance. It does not reject untrusted data as a newly validated boundary. Callers requiring verification must validate before invoking this helper. Supported input is ordinary scalar case data; arbitrary object graphs/getters are outside this contract.4. Security & Privacy Impact
Returned identity/binding objects are detached by explicit structural copies; shared canonical definitions are not frozen. This addresses mutation through results, not direct mutation of separately exported canonical constants. Future nested identity fields require renewed isolation review.
No provider calls, credentials, live experiments, transport changes or empirical evidence imports. Security baseline files from merged PR #182 are preserved exactly, including pnpm 11.11.0. Path sanitization is not a new aggregation guarantee; this helper accepts trusted evaluated data and performs no path-based verification.
5. Scientific Methodology & Behavioral Evidence Impact
qualificationOutcome = NOT_QUALIFIED; public maturity BM2 / UNVALIDATED_PROXY / NOT_PROMOTED;scientificAuthority = NONE.Frozen historical evidence remains unchanged, including the four HTTP 429 infrastructure observations. No qualification recovery, empirical rerun, scientific validation, BM3/BM4/BM5 promotion, leaderboard, Cyber or release. Local regression demonstrates bounded engineering compatibility, not independent scientific replication.
6. Documentation & Guides
The existing pilot guide, function documentation and patch changeset disclose output isolation and the trusted-input contract. Operational integration reports remain outside Git.
7. Migration Impact & Breaking Changes
Intentional API behavior change: reference equality/aliasing with canonical definitions and between results is no longer preserved. Callers must compare identity field values rather than rely on shared object identity or mutate outputs to modify canonical definitions. Outputs remain mutable; immutability is not promised.
No serialized-contract or metric-mathematics breaking change. No new stable public API guarantee is introduced. The changeset accurately records the reference-behavior change.
8. Review Readiness
Draft; independent
@Logorythmus-org/semantiq-maintainersreview remains pending. The protected ruleset requires one approval including CODEOWNER review, stale-review dismissal and resolved review threads. Current required CI is PENDING. Independent human review remains pending.Technical integration is conflict-free and locally validated. CONTRIBUTING.md's up-to-date-branch requirement was addressed by the authorized synchronization with protected main
24c885c0cfc46bd3eb35fc38189397683b47ff59. Fresh CI is being evaluated oncf920d66873cff992daab0314881f9a4795eb664; previous approvals must not be carried across a changed head contrary to stale-review protection.This description update and the bounded branch synchronization were expressly authorized. No reviewer request, Ready transition, approval, merge into main or release is authorized. PR #166 remains Draft and unmerged.