Skip to content

feat(storage): plan retained manifest imports - #609

Draft
fishmingyu wants to merge 3 commits into
feat/retained-manifest-prerequisites-v2from
feat/manifest-import-planner-v2
Draft

feat(storage): plan retained manifest imports#609
fishmingyu wants to merge 3 commits into
feat/retained-manifest-prerequisites-v2from
feat/manifest-import-planner-v2

Conversation

@fishmingyu

@fishmingyu fishmingyu commented Aug 12, 2026

Copy link
Copy Markdown
Member

Summary

Add a pure, backend-neutral planner that converts bounded RepoManifest v1.1 input into immutable retained-import intent. This layer validates and selects portable artifacts, but deliberately does not read artifacts, write CAS objects, mutate the catalog, or publish refs.

Changes

  • Add immutable source, view, selection, and import-plan value types plus mapping and byte-oriented planning entry points.
  • Parse manifest JSON under strict UTF-8, ambiguity, size, depth, node, key, atom, integer, and selection bounds.
  • Admit only current BM25 schema 8 and vector schema 6 artifacts with complete generation and builder-profile identity; required views fail closed while optional views record explicit skip decisions.
  • Treat source fingerprint v2 as eligible input for later authentication and keep v1 plans inert; reject diagnostic and authority claims embedded anywhere in the manifest.
  • Export the planner API and record the retained-manifest planning layer in the storage roadmap.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactoring
  • Performance improvement
  • Tests

Testing

  • Tests pass locally

  • Added new tests for the changes

  • pytest -q test/compiler/test_manifest_storage.py --tb=short — 256 passed

  • pytest -q test/compiler test/storage/test_models.py --tb=short — 616 passed

  • Black 24.8.0, isort 5.13.2 with the repository profile, flake8 7.1.1 plus bugbear, py_compile, and git diff --check pass for the changed surface.

Checklist

  • My code follows the project style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Stacked on #606, which is itself stacked on #605. Keep this PR in Draft until both dependencies are merged or the stack is explicitly restacked and reverified.

Add a pure bounded planner for current RepoManifest v1.1 BM25 and vector projections. The planner freezes source, selection, profile, and generation intent without filesystem, native parser, CAS, catalog, or environment access.

Required views fail closed, optional incompatibilities are explicit, and all public construction and JSON entry points enforce bounded canonical values before copying or traversal.

Verified with 255 focused tests and 615 tests across compiler plus storage models, along with Black, repository-profile isort, flake8, py_compile, and diff checks.
Record the pure RepoManifest planning boundary and keep retained artifact reads, CAS upload, snapshot publication, export, and M2 job publication explicitly outstanding.
@github-actions github-actions Bot added type/docs Documentation type/test Test-related scope/compiler Index compiler, manifest, build integration labels Aug 12, 2026
Propagate the remaining encoded-byte budget through recursive manifest snapshots so object keys, separators, and minimum child size are charged before a caller-owned child mapping is consumed. Bound scalar and container values at each recursion level.

Verified with 256 focused planner tests and 616 tests across compiler plus storage models, including a hostile child mapping that remains unconsumed after the parent budget is exhausted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope/compiler Index compiler, manifest, build integration type/docs Documentation type/test Test-related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant