feat(storage): validate strict publication inputs - #590
Closed
fishmingyu wants to merge 29 commits into
Closed
Conversation
Bind portable BM25 and vector validation to bounded ownership inventories and descriptor-authenticated payloads. Align contained source symlink handling across fingerprints, binding, BM25 queries, and MCP reads. Verified with the artifact/storage/vector/lazy suites, the Python 3.10 unit tier excluding the unavailable Docker-only test file, formatting, flake8, compileall, and the namespace check.
Add canonical per-file ownership records, descriptor-bound readers and owned staging, bounded streaming JSON, process-local native parser authorization, and pinned source fingerprint reads.\n\nVerified with 100 focused unit tests plus flake8 and diff-check.
Bind staged-directory reads and no-replace publication to retained lexical authorities, add authenticated streaming readers and durable Linux commit hooks, and make descriptor/HANDLE cleanup cancellation-safe.\n\nVerified with 141 atomic tests plus independent race, reuse, and durability review.
Publish immutable regular files through retained parent descriptors with no-replace rename, exact authenticated reuse, durable receipts, and retryable cancellation-safe ownership.\n\nVerified with 95 focused tests and the combined atomic publication tier.
Replace hard-link and overwrite fallbacks with owned no-replace publication, pin the preprovisioned lexical shard generation, and replay the full directory durability chain after ambiguous provisioning failures.\n\nVerified with 70 CAS tests, 272 storage tests, and the combined publication tier.
Validate exact workspace plans over caller-provisioned directory authorities, write only planned no-replace files, flush complete trees, and publish Linux generations into caller-owned authenticated receipts.\n\nThe receipt lifecycle retains one authority across validation and consumption and closes cancellation, concurrency, fork, and persistent-cleanup races. Verified with 52 focused tests and two independent reviews.
Bind native vector authorization to the freshly captured tree and semantic contract before constructing embedding models or remote clients. Keep every portable artifact origin parser-inert for eager and lazy loads. Verified with MCP context, pre-model authority, and cleanup-priority tests.
Require exact captured-tree authorization before native parsing in compiler, agent, pipeline, script, and example call sites. Distinguish missing capabilities from malformed or mismatched capabilities so only absence can select a safe source rebuild. Keep local administrative minting at explicit source-bound boundaries and preserve cleanup failures without hiding the first exception.
Build vector files, cache state, and update markers through a descriptor-anchored private generation before one owned-directory switch. Reject generation-root replacement without writing through attacker-controlled paths. Disable live incremental mutation until it can satisfy the same boundary; validated requests take the complete-generation rebuild path and preserve the previous tree under BaseException faults.
Authenticate FAISS payload bytes and canonical metadata without importing native parsers or deserializing pickle. Treat trusted-local as descriptive only and require an exact out-of-band capability for any native normalization. Bind context staging minting to the verified checkout, persisted config fingerprint, and copied tree while preserving first-primary cleanup failures.
Keep the first entry or body BaseException authoritative when POSIX or Windows lock cleanup also fails. Complete descriptor and registry cleanup, retain secondary diagnostics, and propagate cleanup itself when no earlier fault exists.
Require a resolver for hybrid native views unless the caller explicitly chooses optional sparse fallback. Mint local capabilities outside the registry only under the compiler cache lock after exact manifest, source-fingerprint-v2, and captured-tree checks. Keep legacy route defaults out of the capability semantic contract and reject wrong-tree or wrong-config tokens before model construction.
Document parser-inert portable validation, descriptor-anchored whole-generation publication, source-bound Web authorization, and the temporary full-rebuild behavior for vector incremental requests.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1d29793249
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
14 tasks
Bound aggregate workspace-plan path metadata before provisioning and make return-record construction participate in the terminal failure transition. Record the provider-neutral publication foundation in the storage roadmap. Verified with 123 targeted tests plus Black, isort, flake8, py_compile, and git diff --check.
fishmingyu
marked this pull request as draft
August 11, 2026 23:49
14 tasks
fishmingyu
force-pushed
the
fix/native-vector-authority-closure-v2
branch
from
August 13, 2026 21:22
9f6b535 to
f0bfcef
Compare
Base automatically changed from
fix/native-vector-authority-closure-v2
to
main
August 13, 2026 21:35
14 tasks
Member
Author
|
Superseded by #619. The bounded-JSON and credential-classifier slices landed through #616, and the detached workspace identity foundation landed through #586. #619 forward-ports only the remaining provider-neutral workspace publication gate onto current main. The original branch is intentionally retained while the dependent producer work is restacked. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add the bounded parsing and strict workspace validation foundations required by the next portable publication layer.
This is a stacked PR based on #588. It intentionally excludes the later static export, BM25, and context publication migrations so those authority-sensitive changes remain independently reviewable.
Changes
Type of Change
Testing
pytest -q test/test_bounded_json.py test/test_captured_directory.py test/test_native_index_authorization.py test/test_source_fingerprint.py test/storage/test_models.py test/storage/test_view_bundle.py --tb=short(312 passed, 5 skipped)git diff --checkon the changed surfaceChecklist
Dependency: #588 is published and green but intentionally not merged yet.