fix(runtime): retain authenticated source authorities - #587
Conversation
github-actions
Bot
added
type/test
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0cb24e18b6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22136246df
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
29697d2 to
ff0884e
Compare
Bind native vector authorization to the freshly captured tree and semantic contract before constructing embedding models or remote clients. Keep every portable artifact origin parser-inert for eager and lazy loads. Verified with MCP context, pre-model authority, and cleanup-priority tests.
Bind source, archive, and vector consumers to detached authenticated state, preserve retryable native-resource ownership across cancellation, and preflight archive metadata before parser allocation. Also migrate vector callers to explicit authorization, make state publication atomic, and cover hidden-link, fork, descriptor, HANDLE, archive, and consumer cleanup boundaries. Verified with the local unit tier (5010 passed, 71 skipped, 191 deselected; one exact-base umask failure excluded) and all pre-commit hooks.
2213624 to
b856efd
Compare
github-actions
Bot
added
scope/graph
Replace the CPython opcode-offset injection with a narrow method wrapper that raises immediately after the real source ownership transfer. This keeps the cleanup assertion intact across Python 3.12 and xdist scheduling.
Replace CPython opcode-offset injections with narrow ownership seams that raise immediately after the real Windows source or Zoekt process owner takes control. This preserves the cleanup and retry assertions under Python 3.12 and xdist.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 314541ff8e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if self._native_index_authorization is None: | ||
| self.errors["vector"] = ( | ||
| "native vector parsing requires external authorization; " | ||
| "portable artifacts remain inert by default" | ||
| ) | ||
| logger.warning("Vector view is inert without external authorization") | ||
| return |
There was a problem hiding this comment.
When CodeNibBackend.from_manifest(..., index="vector") starts the serving backend, the unchanged call in codenib/serving/drafter/retrieval.py:394 supplies no authorization, so this branch always records an error and returns before constructing the vector store; dense and hybrid case validation similarly calls ServerContext.validate_views() without a token at codenib/eval/benchmarks/policy_compat.py:463-466. Fresh evidence beyond the previously fixed loader list is that these two production entry points remain tokenless, making vector serving fail at startup and vector-required benchmark cases ineligible. Mint and pass authorization at these administrator boundaries, or provide an authorized loading path they can use.
Useful? React with 👍 / 👎.
| def _create_archive_snapshot() -> int: | ||
| try: | ||
| return _create_sealable_memfd() | ||
| except _SnapshotUnavailable as exc: | ||
| raise ValueError( | ||
| "secure context archive parsing requires immutable sealed snapshots" |
There was a problem hiding this comment.
On macOS, os.name is "posix", so archive extraction reaches this helper, but _create_sealable_memfd() is explicitly Linux-only and raises _SnapshotUnavailable when Darwin has no memfd_create; consequently every valid context archive fails before ZIP parsing. Fresh evidence after the Windows-specific fix is that the new Windows HANDLE branch does not cover Darwin even though the package remains classified as OS-independent. Use a bounded immutable Darwin snapshot implementation or a safe platform fallback.
Useful? React with 👍 / 👎.
| with view.authenticated_snapshot(relative) as (snapshot, _record): | ||
| source = AuthenticatedSnapshotReader(snapshot) | ||
| return faiss.read_index(faiss.PyCallbackIOReader(source.read)) |
There was a problem hiding this comment.
On Windows, macOS, or a Linux host where sealed memfds are unavailable, authenticated_snapshot() falls back to immutable in-memory chunks only for files up to 512 MiB; this new path therefore rejects any larger FAISS file before faiss.read_index() even though _vector_entry_policy explicitly admits indexes up to 8 GiB. Such indexes previously loaded directly from disk, so normal large repositories now fail on these supported environments. Provide an authenticated non-memfd streaming or owned-file fallback for the native parser instead of routing every FAISS file through the 512 MiB snapshot ceiling.
Useful? React with 👍 / 👎.
| if self.source_mode == SOURCE_MODE_PERSISTED_ONLY: | ||
| selected = _split_source_lines(doc.page_content) | ||
| start_idx = max(0, start_line) |
There was a problem hiding this comment.
In query-only artifact contexts, this branch treats doc.page_content as source text, but chunk documents were already transformed by _apply_stemming() when the BM25 index was built: identifiers are lowercased, punctuation and newlines are removed, and calls such as foo() become plain foo. Consequently mixed-case exact identifier queries produce false negatives, invocation counts are lost, and returned snippets are tokenized text mislabeled with the original source line range. Persist raw chunk text for this operation, or leave exact occurrence scanning unavailable when no authenticated source bytes exist.
Useful? React with 👍 / 👎.
1 participant
Summary
Retain authenticated source, archive, vector, and runtime authorities across production consumers so mutable paths and interrupted cleanup cannot silently replace verified bytes, leak native resources, or publish partially validated state.
This branch is rebased directly onto current
mainafter #586 and #617 merged.Changes
ZipFileallocation, and retain cleanup owners across malformed-input translation.Type of Change
Testing
Tests pass locally
Added new tests for the changes
Post-rebase focused surface:
935 passed, 16 skipped, 11 deselected.Post-rebase unit tier:
5010 passed, 71 skipped, 191 deselected.The only manually deselected case is an exact-base cache-lock/umask failure reproduced on the prior clean base; it creates mode
0775under the local0002umask.All repository pre-commit hooks and
git diff --checkpassed.Python source/bytecode compatibility, updated atomic cleanup annotations, ZIP envelope preflight, vector consumers, and Zoekt lifecycle tests are included in the focused result.
Checklist
Dependencies #584, #585, #586, and the extracted atomic-cleanup replacement #617 are merged into
main; the branch was rebased after those merges.