Skip to content

Complete Pines hardening, product UX, and exact-pair validation#14

Draft
RNT56 wants to merge 58 commits into
tq/integration-pin-mlx-productionfrom
tq/real-device-evidence-acceptance
Draft

Complete Pines hardening, product UX, and exact-pair validation#14
RNT56 wants to merge 58 commits into
tq/integration-pin-mlx-productionfrom
tq/real-device-evidence-acceptance

Conversation

@RNT56

@RNT56 RNT56 commented May 25, 2026

Copy link
Copy Markdown
Owner

Summary

  • Hardens the local-first data plane with SQLCipher migrations, bounded provider and MCP ingestion, SSRF-safe networking, CloudKit lifecycle repair, persisted MCP safety annotations, and provider-scoped catalog snapshots.
  • Completes the responsive iOS, iPadOS, watchOS, and public-site design pass across Chats, Models, Vault, Artifacts, Settings, project spaces, navigation, accessibility, and operational empty states.
  • Expands OpenRouter from a generic compatible endpoint into typed routing/privacy controls, structured-output enforcement, route and cost receipts, capability-driven model selection, encrypted six-hour catalog freshness, and bounded beta server web search with engine selection, public citations, usage, and hosted-tool provenance.
  • Advances the maintained MLX fork pair to mlx-swift bcf93af23f11428f6f01efb0bb4b9020cd2eb383 and mlx-swift-lm aeaa8e3024a82b25969741b53c749b28ddc64d1a with the compatibility metadata and diagnostics needed for exact-pair evidence.
  • Upgrades the Astro site and dependency graph. The current branch reports zero npm audit vulnerabilities; default-branch Dependabot alerts remain visible until this stack reaches main.

Current validation

  • swift test --disable-automatic-resolution: 235 tests in 15 suites pass.
  • swift run --disable-automatic-resolution PinesCoreTestRunner: all checks pass.
  • Focused iOS Simulator OpenRouter request/settings and app-surface suite: 34 tests pass.
  • Unsigned generic iOS app build passes from the earlier branch-wide validation.
  • Public repository hygiene, privacy manifest, high-assurance security boundaries, secret scan, generated-artifact checks, third-party notices, action pins, and MLX pin alignment pass locally.
  • Site npm audit reports zero known vulnerabilities.
  • CI and CodeQL for the current head are pending and must be green before merge.

OpenRouter server-search boundary

Pines now maps provider-neutral automatic/required web search to OpenRouter's beta openrouter:web_search server tool, including explicit engine selection, bounded result/domain/location policy, fail-closed external-web access, public-URL citation ingestion, request-count accounting, source presentation, and hosted-tool timeline entries. The request, parser, persistence, settings, and UI contracts are covered locally; a live provider call still requires a user-configured OpenRouter key, incurs provider charges, and remains model/route dependent.

Real-device evidence and product boundary

A small real-model comparison completed on an iPhone 15 Pro Max with the installed mlx-community/Qwen3.5-0.8B-MLX-4bit model at 4K context, 8 output tokens, and two repeats: 1/1 scenario passed; FP16 median decode was 42.279 tok/s; affine K8/V4 median decode was 42.373 tok/s; the quality gate passed with top-1 agreement 1.0; and no raw fallback was used.

That installed Pines binary used an older immutable MLX compatibility pair, not this branch's current bcf93af + aeaa8e3 pair. The result is therefore useful historical runtime smoke only: it does not satisfy current-pair acceptance, import a product evidence tuple, or justify any Verified/Certified, native performance parity, snapshot-restore promotion, or full A16-through-A19 Pro acceptance claim. Exact-current-pair native backend, full benchmark matrix, quality, memory, fallback, and accepted physical-device evidence remain release gates.

RNT56 commented May 25, 2026

Copy link
Copy Markdown
Owner Author

Real-device TurboQuant smoke run completed on the physical iPhone target.

Device:

  • Name: GBU-12
  • Model: iPhone 15 Pro Max (iPhone16,2)
  • Device ID: 00008130-00041C6E2EB8001C
  • iOS: 26.5 (23F77)
  • Architecture: arm64

Command:

xcodebuild -project Pines.xcodeproj \
  -scheme Pines \
  -destination 'platform=iOS,id=00008130-00041C6E2EB8001C' \
  -derivedDataPath build/DerivedDataDevice \
  -skipMacroValidation \
  -skipPackagePluginValidation \
  -onlyUsePackageVersionsFromResolvedFile \
  -disableAutomaticPackageResolution \
  -scmProvider system \
  -allowProvisioningUpdates \
  ONLY_ACTIVE_ARCH=YES \
  '-only-testing:PinesTests/MLXTurboQuantRuntimeSmokeTests' \
  '-skip-testing:PinesUITests' \
  test

Result:

  • MLXTurboQuantRuntimeSmokeTests: 10 tests passed, 0 failed
  • Physical-device Metal codec path covered by testHighBitSeedMetalCodecRoundTripWhenAvailable
  • Fixed high-bit seed device path covered by testTurboQuantCacheUsesFixedHighBitSeedOnDevice
  • Result bundle: build/DerivedDataDevice/Logs/Test/Test-Pines-2026.05.25_22-24-29-+0200.xcresult

Follow-up committed in 32e9ca9: hosted PinesTests on the app target because physical iOS devices cannot run tool-hosted XCTest bundles.

Scope note: this is a targeted real-device smoke pass, not a full BenchmarkReport.v1 acceptance tuple. It should not promote any model to Verified/Certified by itself.

RNT56 and others added 28 commits May 30, 2026 09:43
Use plain FP16 KV (faster, higher quality) whenever its uncompressed cache fits
the live memory budget; fall to TurboQuant only to reach contexts that otherwise
would not fit RAM — replacing the static min(ctx, 8192) plain-KV cap with a
memory-feasibility decision.

- MLXRuntimeBridge.kvCacheAdmission honors admission.recommendsPlainKVCache:
  returns plain FP16 at full admitted length (no 8K cap); conversion carries the
  flag through coreTurboQuantAdmission.
- LocalRuntimeAdmissionService.admit: FP16-first ladder (fp16KVBytesPerToken;
  FP16-full → [.fastest: shorter FP16] → TurboQuant) + recommendsPlainKVCache on
  the PinesCore type. +7 ladder tests (206 PinesCoreTests pass).
- Bump mlx-swift (aa4a071: cooperative coalesced QK decode, opt-in) and
  mlx-swift-lm (002ec99: recommendsPlainKVCache planner) pins.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-place KV-cache donation fix on append (was reallocating full-capacity buffers
per token — audit 1.3 OOM suspect). 69 cache tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
4.1: fallback decode uses fp16 scratch + an OOM guard (recoverable instead of crash).
2A: mid-generation FP16->compressed spill under memory pressure (the missing dynamic
half) — GenerateParameters.spillMemoryWatermarkBytes, default off (on-device tuning).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Picks up the mlx-swift-lm fork tip carrying the turbo3 bit-metadata fix (5.5)
and the new TurboQuantBench on-device A-series benchmark harness. Both are
behavior-neutral / test-only for the app (pines uses its own scheme enum without
turbo3; the app does not depend on the TurboQuantBench product), so this is a
sync + manifest-resolution bump.

Verified: full Pines app resolves mlx-swift-lm @ 3118d5b and builds green
(xcodebuild, iOS Simulator, -skipPackagePluginValidation -skipMacroValidation).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures the working-tree TurboQuant work (control-plane + evidence types, tests,
and the turboquant-implementation docs/baselines) as a green checkpoint at the
current MLX pin pair (mlx-swift 609e833 + mlx-swift-lm 725add5). PinesCore builds
and all 227 PinesCore tests pass, including TurboQuantPinDriftTests.

Note: the mlx-swift-lm pin bump to pick up the N2 self-speculation API
(makeGenerationIterator + GenerateParameters.selfSpeculationMode) is intentionally
NOT included here — it requires regenerating compatibility-pair.json via the
validation harness (the evidence artifact must not be hand-edited), which needs the
deferred A-series device run for full evidence. See the mlx-swift-lm overhaul
handoff (N2 Pines section) for the exact pin-coordination sequence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents that mlx-swift-lm 295e66b now exposes the N2 self-speculation product
API (GenerateParameters.selfSpeculationMode + makeGenerationIterator, bit-exact,
default-off) and mlx-swift adds the data-free Gaussian payload codec, and that
adopting them requires advancing the MLX pin pair + regenerating compatibility-
pair.json via the wave0 harness (not hand-edited) + the deferred A-series device
run. Self-speculation ships default-off (inert until enabled + device-validated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DEBUG-only, launch-gated (PINES_TQ_REAL_BENCH=1) real-model device benchmark: loads the
ACTUAL on-device model weights via the production loader and runs real token generation
comparing compressed affineK8V4 vs plain FP16 KV through the canonical
InferenceParityBenchmark engine (real-model-inference-v1) — the real-model throughput +
cosine/top-1 quality evidence the promotion gate requires (the synthetic TurboQuantBench
sweep cannot provide it). Writes JSON to Documents/PinesDiagnostics.

- PinesRealModelTurboQuantDiagnostics.swift (new): per-context isolation (a 64K OOM doesn't
  lose 32K), bootstrap 95% CIs on each arm's median decode tok/s AND on the compressed/FP16
  ratio, multi-model support (PINES_TQ_REAL_MODELS csv) so a smaller fallback model can cover
  long contexts a larger model's weights+KV won't fit. Ensure-downloads via the app's own
  ModelLifecycleService (idempotent).
- project.yml + generated pbxproj: add the IntegrationTestHelpers library product dependency.
- PinesRootView: invoke the hook at launch alongside the existing DEBUG bench hooks.

DEBUG-gated, inert in release. Build on device with -jobs 6 -skipPackagePluginValidation
-skipMacroValidation. First result (A17 Pro, real Qwen3.5-2B-OptiQ-4bit): 16K compressed
0.96x FP16 with byte-identical greedy output. Evidence lives in the mlx-swift-lm repo.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Finish the persistence and CloudKit lifecycle, Project Spaces synchronization, bounded provider and MCP response handling, SSRF-safe redirect policy, and conservative tool-safety persistence.

Advance the maintained MLX dependency pair to the platform-aware Apple-mobile-compatible revisions, upgrade the Astro site and CI security gates, and synchronize project, runtime, documentation, and compatibility metadata.

Record exact-pair iPhone synthetic and Qwen 3.5 0.8B real-model smoke evidence with explicit non-promotion caveats, backed by core, iOS, UI, security, and site validation.
Remove legacy trailing whitespace surfaced by the pull-request base comparison so the repository hygiene job evaluates the full branch cleanly.
Support both legacy and platform-derived MLX Metal helper assignments, keep the tracer bypass idempotent, and fail closed when an upstream helper can no longer be patched.
Preserve cloud-provider identity and credentials across edits, expose CloudKit sync health and retry, and add typed OpenRouter routing, privacy, and structured-output controls. Cover the new request and lifecycle semantics with package and app tests, and reconcile provider documentation with the shipped behavior.
Boot and wait for a clean simulator before each CI test phase, bound xcodebuild execution, and retry once after launcher failures so hosted runner stalls fail fast instead of consuming the full job timeout.
Unify responsive, accessible design primitives and content-first layouts across Chats, Models, Vault, Artifacts, Settings, and watchOS. Add safer actions, actionable empty states, compact controls, and broader UI coverage including accessibility sizing.
Complete the end-to-end Pines design upgrade
Finalize provider metadata after terminal accounting chunks so streamed usage and routing details are not lost. Persist a bounded privacy-minimized OpenRouter receipt, expose it through progressive chat disclosure, and reconcile provider status documentation.
Retain bounded pricing, modality, context, and lifecycle metadata from the OpenRouter catalog.

Reject known-incompatible requests before spend and expose searchable model details in the chat picker.

Cover metadata compatibility and eligibility behavior, and reconcile provider status documentation.
Store bounded provider catalog snapshots in encrypted GRDB with six-hour expiry and provider-deletion cascade.

Hydrate fresh snapshots at launch, retain them across transient refresh failures, and keep expired OpenRouter metadata out of hard request rejection.

Cover schema, bounds, expiry, round-trip persistence, replacement, cleanup, and verifier invariants.
@RNT56 RNT56 changed the title Pin Layout V5 default MLX pair for device evidence Complete Pines hardening, product UX, and exact-pair validation Jul 13, 2026
Add bounded router-hosted web search requests and persisted engine controls.

Normalize citations and usage into source panels, hosted-tool timelines, and run receipts with request, parser, settings, and compatibility coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant