Skip to content

Harden native memory sidecars, signals, cache, and CI - #3

Merged
NGO-A merged 16 commits into
masterfrom
security/nmc-hardening-r2
Aug 2, 2026
Merged

Harden native memory sidecars, signals, cache, and CI#3
NGO-A merged 16 commits into
masterfrom
security/nmc-hardening-r2

Conversation

@NGO-A

@NGO-A NGO-A commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Verification Report: native-memory-citations hardening round 2

Date: 2026-08-01 MDT
Branch: security/nmc-hardening-r2
Base: security/native-memory-citations-boundary-perf-hardening at 4855a15
Head: 6c7409b

Stacked PR: Retarget to master after PR #2 merges.

Summary

Implements the full 12-item hardening round: immutable release pins; OS/Node CI,
typecheck, coverage, failure injection and tarball smoke; stored-orientation graph
traversal and graph caching; charset-aware redaction; versioned atomic sidecars and
write boundaries; explicit output-bound signals; fail-closed tool registration;
64 MiB byte-budget LRU; DREAMS.md default scope; health and documentation updates.

Decision gates

  • G1: implemented scan + documented envelope. Cache budget is 64 MiB; README envelope is <=500 files / <=50 MB. FTS5 remains outside this plugin.
  • G2: added DREAMS.md to private default roots. Release-note draft: artifacts/handoffs/release-note-draft-nmc-dreams-default.md.
  • G3: drafted only; not filed. Draft: artifacts/handoffs/upstream-sdk-registration-issue-draft.md.

Required reproductions

Reproduction Result
200 generated 64-character hex secrets 200/200 redacted; charset-aware entropy redaction > redacts all 200... passes.
Stored Alice works_at Acme, queried from Acme Stored orientation returned with direction: "reverse"; no inverted assertion; regression passes.
One corrupt graph JSONL line Query succeeds with one edge, skippedLines: 1, and warning; regression passes.
Fetch line 500 from a 100-line file contract Out-of-range starts throw and include the requested start and actual line count; regression passes.

Additional evidence

  • Symlink write escape: graph and snapshot writers both reject a memory symlink outside the workspace before writing; no outside sidecar is created.
  • Snapshot ordering: a secret straddling the token slice is redacted before slicing; no tested secret prefix remains.
  • Registration boundary: unknown SDK conveyance shape is dropped in bounded mode, warning emitted, and health-visible counter incremented; enhanced mode passes it through.
  • Cache cliff: a 513-file fixture plus the two standard fixture files performs 515 disk loads on the first query and 0 on the second unchanged query.
  • Cache validity: same-size replacement with preserved mtime is refreshed via ctime/inode mismatch.
  • Cache envelope: over-budget scans set degraded, include degradedReason, and expose health state.
  • Atomic/failure injection: simulated ENOSPC and interrupted rename leave the original sidecar intact; simulated EACCES increments skippedFiles while readable hits return.
  • Concurrency: two parallel graph extracts plus an extract-time query complete with a valid versioned sidecar.
  • Tarball smoke: npm pack, install into a scratch project, import packaged entry, register the bounded surface, and invoke native_memory_search passed.

Test and build evidence

Command Result
npm run test:typecheck Passed.
npm test Passed: 141/141 tests, 6 files.
npm run test:stress Passed: 14/14 tests.
npm run test:coverage Passed: statements 88.66%, branches 87.21%, functions 90.83%, lines 88.66%; enforced floors 88/85/90/88.
npm run test:pack Passed; packed ngo-a-native-memory-citations-2026.6.18.tgz in a temporary directory. No version was changed.
npm run plugin:build:check Passed; generated metadata current.
Node 24 (v24.18.1) test typecheck + full Vitest Passed: 141/141.
git diff --check Passed.

Local plugin build output retained pre-existing environment warnings: the project-local OpenClaw CLI is 2026.6.9 while the active host config was written by 2026.7.1-2, plus existing duplicate/missing optional plugin notices. Metadata validation still passed.

CI matrix status

The workflow now defines Ubuntu/macOS/Windows x Node 22.19.0/24.x x OpenClaw 2026.6.8/2026.6.9. Ubuntu behavior was verified locally on the installed Node 22 line and separately with Node 24. macOS and Windows remain explicitly non-blocking because the stacked PR targets a feature branch and this repository's pull_request.branches filter runs only for master/main; hosted jobs cannot execute until the PR is retargeted after PR #2 merges. No test was silenced.

Action pin mapping

  • actions/checkout v6 -> d23441a48e516b6c34aea4fa41551a30e30af803
  • actions/setup-node v6 -> 249970729cb0ef3589644e2896645e5dc5ba9c38
  • actions/upload-artifact v7 -> 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
  • Existing reusable ClawHub workflow remains pinned to 6f28659e7bfb5ae21e1bcc2d6896fea9c0a7f698.
  • Grep for mutable action refs (@v*, @main, @master, @latest) returns no matches.

Anchor audit

All quoted production-code anchors were located on the PR #2 base. No work order was stopped for a missing anchor. Two review assertions differed from the base and were handled explicitly:

  • PR Bound native memory worst-case hardening costs #2 introduced cost caps but no existing truncated/capped output fields, so round 2 established the documented naming contract.
  • The two cited graph tests checked depth/cycle behavior but did not pin the inverted edge shape; the new exact regression was added without deleting those checks.

Scope and owner gates

All 12 listed work orders were implemented. No runtime dependency was added. No version was stamped. No tag, npm/ClawHub publish, deprecation, GHSA, merge, PR #2 rewrite/rebase, or reporter attribution occurred. Out-of-scope review items were not implemented.

Changelog draft (Unreleased)

Changed

  • Compatibility: graph path edges preserve stored orientation and add direction.
  • Compatibility: out-of-range fetch starts now throw with the actual line count.
  • Compatibility: search tool details use the documented diagnostic envelope.
  • Entropy redaction covers long hex and slash-bearing alphabets and wider key names;
    full hashes/checksums may now be over-redacted by design.
  • DREAMS.md joins private default roots and enhanced snapshot eligibility.
  • Bounded mode drops unknown SDK registration shapes.
  • File caching is a 64 MiB true LRU with a <=500-file / <=50-MB documented envelope.

Security

  • Sidecar writes validate real workspace containment and use temporary replacement.
  • Graph/snapshot formats are versioned; legacy graph files remain readable, corrupt
    graph lines are counted/skipped, corrupt snapshots warn and degrade to absent.
  • Snapshot redaction occurs before token slicing.

CI

  • Actions are pinned by SHA; npm@latest is removed from publishing.
  • OS/Node matrix, test typecheck, branch-coverage floor, failure injection, and packed
    artifact smoke coverage are added.

NGO-A added 12 commits August 1, 2026 08:02
Pin actions/checkout v6 to d23441a48e516b6c34aea4fa41551a30e30af803, actions/setup-node v6 to 249970729cb0ef3589644e2896645e5dc5ba9c38, and actions/upload-artifact v7 to 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a. Remove the unbounded npm@latest install from the publish job.
Add test-inclusive TypeScript checks and V8 coverage with floors just below the measured baseline (88.88% statements, 85.97% branches, 91.58% functions, 88.88% lines). Coverage enforcement is not wired into CI until the final failure-injection commit.
Run the OpenClaw compatibility matrix across Ubuntu, macOS, and Windows on Node 22.19.0 and 24.x. Keep new macOS and Windows lanes non-blocking until their first runs establish whether any failures are pre-existing.
Add directional traversal metadata without mutating stored edges, replace quadratic BFS operations, hoist graph extraction regexes, simplify single-hit answer selection, and cache parsed graph materialization by mtime/size/ctime/inode.
Detect 32+ character hex candidates at 3.4 bits per character, retain the 4.0-bit threshold for mixed/base64 candidates, exempt numeric IDs and multi-segment paths, keep slash-bearing AWS-style candidates intact, and widen named credential keys.
Add versioned graph and snapshot sidecars with legacy reads, tolerant graph-line parsing, corrupt snapshot fallback, atomic temporary-file replacement, and workspace-contained write-parent validation. Share graph path, dreaming detection, and host config derivations across modules.
Reject out-of-range fetch starts with the actual line count; mark fetches and search hits truncated; return search tool diagnostics for skipped files and capped queries/results; surface corrupt graph line counts and graph extraction caps.
Recognize only the two current SDK tool conveyance shapes in bounded mode, warn and record a health-visible finding for unknown shapes, preserve enhanced passthrough, and replace broad registration casts with explicit SDK-derived types.
Replace the 512-entry cliff with a 64 MiB true LRU, account raw/redacted line storage, drop retained rawText, validate cache entries with mtime/size/ctime/inode, and signal scans beyond the documented cache envelope.
Add DREAMS.md to private defaults, warn when explicit roots exclude MEMORY.md, document bounded output signals and the 500-file/50-MB scan envelope, explain deterministic graph timestamps, and record all compatibility/security changes under Unreleased with CalVer semantics.
Cover ENOSPC, interrupted rename, EACCES, parallel extracts, and extract-during-query. Enforce measured coverage floors on the canonical CI tuple and install the npm tarball into a scratch project before invoking a bounded tool.
@NGO-A
NGO-A force-pushed the security/nmc-hardening-r2 branch from 5d49968 to 6c7409b Compare August 1, 2026 14:30
@NGO-A
NGO-A changed the base branch from security/native-memory-citations-boundary-perf-hardening to master August 2, 2026 11:39
@NGO-A
NGO-A marked this pull request as ready for review August 2, 2026 11:52
@NGO-A
NGO-A merged commit a206aaa into master Aug 2, 2026
13 checks passed
@NGO-A

NGO-A commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

REPORT — Full merge sequence (all phases), @ngo-a/native-memory-citations

Date: 2026-08-02
Authorization: artifacts/handoffs/authorization-2026-08-01-nmc-merge-sequence-all-phases.md
Status: Completed through Phase 4; hard stop active. This was not a release.

Merge record

All three PRs are merged with true merge commits:

PR Merge commit First parent Second parent / merged head
#2 de224446b392a426a3e88802ab238d2c12692d0d becb6fa1e0ca6d0a8238c566de1383aac6afc74b 4855a15b201e2172b6c4534ea3b72c8f04c68a97
#3 a206aaa7ea069110a29fb0e441f5626e7c1c9ebc de224446b392a426a3e88802ab238d2c12692d0d ae61e7eca717bc50b9f6da1ac644ee09adaa72d3
#4 a9a99438c310f3d1b880e5318cabdb24e9b4fcff a206aaa7ea069110a29fb0e441f5626e7c1c9ebc fccea364625f9b20d6b2da475b4521961f1b3fb5

PR #3 recomputed to exactly the 12 authorized round-2 commits before fixes. PR #4 recomputed to exactly the three authorized round-3 commits (c2dd1eb, 13462e2, e4423da) before its test-only successor. Both merge trees were conflict-free; no rebase, force-push, squash, or history rewrite occurred.

Matrix evidence

Each run used the 12 named tuples: Ubuntu/macOS/Windows × Node 22.19.0/24.x × OpenClaw 2026.6.8/2026.6.9.

PR #3 first runs and finals

  • Run 30746220794, head 310af48: Ubuntu 4/4 success; macOS 0/4; Windows 0/4. macOS/Windows were non-blocking on this first run. All eight failures were the same false workspace-escape finding caused by canonical path spelling for missing default roots.
  • Run 30746321594, head 112f5be: Ubuntu 4/4 and macOS 4/4 success; Windows 3/4 success. Windows Node 22.19.0 + OpenClaw 2026.6.9 failed with EPERM during concurrent graph-sidecar replacement.
  • Run 30746434982, head 413d9ea: all 12/12 success, including Windows 4/4.
  • Blocking final run 30746542275, head ae61e7e: all 12/12 success after removing non-blocking semantics.
  • Post-merge master run 30746645489, merge a206aaa: all 12/12 success.

PR #4 and final master

  • PR run 30746766559, head fccea36: all 12/12 success, including the round-3 guard and per-agent override test on macOS/Windows.
  • Post-merge master run 30746863178, merge a9a9943: all 12/12 success.

GitHub Actions may display the first two PR #3 workflow conclusions as success because macOS/Windows were still continue-on-error; the per-job outcomes above preserve the actual lane results verbatim.

Fix and closure commits

Commit Purpose Evidence
310af48e9a40547185c2b88f78048d20c01e0d79 Moved pull_request from invalid permissions placement into the workflow trigger so the hosted matrix could run. All subsequent PR matrices instantiated 12 tuples.
112f5be71e42ce6efc6d430f7b0df2fa0970a7c3 Canonicalized missing default roots relative to the real workspace path across macOS/Windows. macOS 4/4 green on the next run; named symlink/canonicalization test added.
413d9ea7d03223989df99139b353904453158439 Serialized concurrent atomic writes per sidecar path, preserving Windows backup/restore behavior. Windows 4/4 green on the next and every final run; concurrent-replacement test added.
ae61e7eca717bc50b9f6da1ac644ee09adaa72d3 Removed the macOS/Windows non-blocking guard. Blocking final PR #3 matrix 12/12 green.
fccea364625f9b20d6b2da475b4521961f1b3fb5 Test-only closure of the delegated per-agent override review item. Test name: warns and reports health when a per-agent override re-enables host injection; asserts both warning and health finding; full matrix 12/12 green.

Branch protection

master branch protection is active. strict is false so the audited stacked heads did not require an extra base-merge commit. All 12 exact matrix contexts are required:

  • build-validate-test (ubuntu-latest, 22.19.0, 2026.6.8)
  • build-validate-test (ubuntu-latest, 22.19.0, 2026.6.9)
  • build-validate-test (ubuntu-latest, 24.x, 2026.6.8)
  • build-validate-test (ubuntu-latest, 24.x, 2026.6.9)
  • build-validate-test (macos-latest, 22.19.0, 2026.6.8)
  • build-validate-test (macos-latest, 22.19.0, 2026.6.9)
  • build-validate-test (macos-latest, 24.x, 2026.6.8)
  • build-validate-test (macos-latest, 24.x, 2026.6.9)
  • build-validate-test (windows-latest, 22.19.0, 2026.6.8)
  • build-validate-test (windows-latest, 22.19.0, 2026.6.9)
  • build-validate-test (windows-latest, 24.x, 2026.6.8)
  • build-validate-test (windows-latest, 24.x, 2026.6.9)

Release-boundary verification

  • Remote tag baseline and final state both end at existing v2026.6.18; no tag was created or pushed by this sequence. Final complete tag-list SHA-256: f07586ea731925638022956eff34afc667620fa90396e5a34f7480e1969aa684.
  • npm still serves @ngo-a/native-memory-citations@2026.6.18, the same version recorded before Phase 1; nothing was published.
  • CHANGELOG.md on final master keeps all merged work under ## [Unreleased]; no new version was stamped.
  • No ClawHub action, npm publish, deprecation, GHSA action, upstream filing, Notion edit, reporter attribution, or runtime dependency change was performed.

Hard stop / owner-only remainder

The merge sequence is complete and stopped. Remaining owner-only work is unchanged: any later release decision and release steps; the five staged 2026.6.17 steps in their standing order; filing both upstream issues; and the Notion threat-model mapping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant