Complete Pines hardening, product UX, and exact-pair validation#14
Draft
RNT56 wants to merge 58 commits into
Draft
Complete Pines hardening, product UX, and exact-pair validation#14RNT56 wants to merge 58 commits into
RNT56 wants to merge 58 commits into
Conversation
Owner
Author
|
Real-device TurboQuant smoke run completed on the physical iPhone target. Device:
Command: xcodebuild -project Pines.xcodeproj \
-scheme Pines \
-destination 'platform=iOS,id=00008130-00041C6E2EB8001C' \
-derivedDataPath build/DerivedDataDevice \
-skipMacroValidation \
-skipPackagePluginValidation \
-onlyUsePackageVersionsFromResolvedFile \
-disableAutomaticPackageResolution \
-scmProvider system \
-allowProvisioningUpdates \
ONLY_ACTIVE_ARCH=YES \
'-only-testing:PinesTests/MLXTurboQuantRuntimeSmokeTests' \
'-skip-testing:PinesUITests' \
testResult:
Follow-up committed in Scope note: this is a targeted real-device smoke pass, not a full |
added 27 commits
May 25, 2026 23:20
Use plain FP16 KV (faster, higher quality) whenever its uncompressed cache fits the live memory budget; fall to TurboQuant only to reach contexts that otherwise would not fit RAM — replacing the static min(ctx, 8192) plain-KV cap with a memory-feasibility decision. - MLXRuntimeBridge.kvCacheAdmission honors admission.recommendsPlainKVCache: returns plain FP16 at full admitted length (no 8K cap); conversion carries the flag through coreTurboQuantAdmission. - LocalRuntimeAdmissionService.admit: FP16-first ladder (fp16KVBytesPerToken; FP16-full → [.fastest: shorter FP16] → TurboQuant) + recommendsPlainKVCache on the PinesCore type. +7 ladder tests (206 PinesCoreTests pass). - Bump mlx-swift (aa4a071: cooperative coalesced QK decode, opt-in) and mlx-swift-lm (002ec99: recommendsPlainKVCache planner) pins. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-place KV-cache donation fix on append (was reallocating full-capacity buffers per token — audit 1.3 OOM suspect). 69 cache tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
4.1: fallback decode uses fp16 scratch + an OOM guard (recoverable instead of crash). 2A: mid-generation FP16->compressed spill under memory pressure (the missing dynamic half) — GenerateParameters.spillMemoryWatermarkBytes, default off (on-device tuning). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Picks up the mlx-swift-lm fork tip carrying the turbo3 bit-metadata fix (5.5) and the new TurboQuantBench on-device A-series benchmark harness. Both are behavior-neutral / test-only for the app (pines uses its own scheme enum without turbo3; the app does not depend on the TurboQuantBench product), so this is a sync + manifest-resolution bump. Verified: full Pines app resolves mlx-swift-lm @ 3118d5b and builds green (xcodebuild, iOS Simulator, -skipPackagePluginValidation -skipMacroValidation). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures the working-tree TurboQuant work (control-plane + evidence types, tests, and the turboquant-implementation docs/baselines) as a green checkpoint at the current MLX pin pair (mlx-swift 609e833 + mlx-swift-lm 725add5). PinesCore builds and all 227 PinesCore tests pass, including TurboQuantPinDriftTests. Note: the mlx-swift-lm pin bump to pick up the N2 self-speculation API (makeGenerationIterator + GenerateParameters.selfSpeculationMode) is intentionally NOT included here — it requires regenerating compatibility-pair.json via the validation harness (the evidence artifact must not be hand-edited), which needs the deferred A-series device run for full evidence. See the mlx-swift-lm overhaul handoff (N2 Pines section) for the exact pin-coordination sequence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents that mlx-swift-lm 295e66b now exposes the N2 self-speculation product API (GenerateParameters.selfSpeculationMode + makeGenerationIterator, bit-exact, default-off) and mlx-swift adds the data-free Gaussian payload codec, and that adopting them requires advancing the MLX pin pair + regenerating compatibility- pair.json via the wave0 harness (not hand-edited) + the deferred A-series device run. Self-speculation ships default-off (inert until enabled + device-validated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DEBUG-only, launch-gated (PINES_TQ_REAL_BENCH=1) real-model device benchmark: loads the ACTUAL on-device model weights via the production loader and runs real token generation comparing compressed affineK8V4 vs plain FP16 KV through the canonical InferenceParityBenchmark engine (real-model-inference-v1) — the real-model throughput + cosine/top-1 quality evidence the promotion gate requires (the synthetic TurboQuantBench sweep cannot provide it). Writes JSON to Documents/PinesDiagnostics. - PinesRealModelTurboQuantDiagnostics.swift (new): per-context isolation (a 64K OOM doesn't lose 32K), bootstrap 95% CIs on each arm's median decode tok/s AND on the compressed/FP16 ratio, multi-model support (PINES_TQ_REAL_MODELS csv) so a smaller fallback model can cover long contexts a larger model's weights+KV won't fit. Ensure-downloads via the app's own ModelLifecycleService (idempotent). - project.yml + generated pbxproj: add the IntegrationTestHelpers library product dependency. - PinesRootView: invoke the hook at launch alongside the existing DEBUG bench hooks. DEBUG-gated, inert in release. Build on device with -jobs 6 -skipPackagePluginValidation -skipMacroValidation. First result (A17 Pro, real Qwen3.5-2B-OptiQ-4bit): 16K compressed 0.96x FP16 with byte-identical greedy output. Evidence lives in the mlx-swift-lm repo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Finish the persistence and CloudKit lifecycle, Project Spaces synchronization, bounded provider and MCP response handling, SSRF-safe redirect policy, and conservative tool-safety persistence. Advance the maintained MLX dependency pair to the platform-aware Apple-mobile-compatible revisions, upgrade the Astro site and CI security gates, and synchronize project, runtime, documentation, and compatibility metadata. Record exact-pair iPhone synthetic and Qwen 3.5 0.8B real-model smoke evidence with explicit non-promotion caveats, backed by core, iOS, UI, security, and site validation.
Remove legacy trailing whitespace surfaced by the pull-request base comparison so the repository hygiene job evaluates the full branch cleanly.
Support both legacy and platform-derived MLX Metal helper assignments, keep the tracer bypass idempotent, and fail closed when an upstream helper can no longer be patched.
Preserve cloud-provider identity and credentials across edits, expose CloudKit sync health and retry, and add typed OpenRouter routing, privacy, and structured-output controls. Cover the new request and lifecycle semantics with package and app tests, and reconcile provider documentation with the shipped behavior.
Boot and wait for a clean simulator before each CI test phase, bound xcodebuild execution, and retry once after launcher failures so hosted runner stalls fail fast instead of consuming the full job timeout.
Unify responsive, accessible design primitives and content-first layouts across Chats, Models, Vault, Artifacts, Settings, and watchOS. Add safer actions, actionable empty states, compact controls, and broader UI coverage including accessibility sizing.
Complete the end-to-end Pines design upgrade
Finalize provider metadata after terminal accounting chunks so streamed usage and routing details are not lost. Persist a bounded privacy-minimized OpenRouter receipt, expose it through progressive chat disclosure, and reconcile provider status documentation.
Retain bounded pricing, modality, context, and lifecycle metadata from the OpenRouter catalog. Reject known-incompatible requests before spend and expose searchable model details in the chat picker. Cover metadata compatibility and eligibility behavior, and reconcile provider status documentation.
Store bounded provider catalog snapshots in encrypted GRDB with six-hour expiry and provider-deletion cascade. Hydrate fresh snapshots at launch, retain them across transient refresh failures, and keep expired OpenRouter metadata out of hard request rejection. Cover schema, bounds, expiry, round-trip persistence, replacement, cleanup, and verifier invariants.
Add bounded router-hosted web search requests and persisted engine controls. Normalize citations and usage into source panels, hosted-tool timelines, and run receipts with request, parser, settings, and compatibility coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bcf93af23f11428f6f01efb0bb4b9020cd2eb383and mlx-swift-lmaeaa8e3024a82b25969741b53c749b28ddc64d1awith the compatibility metadata and diagnostics needed for exact-pair evidence.Current validation
swift test --disable-automatic-resolution: 235 tests in 15 suites pass.swift run --disable-automatic-resolution PinesCoreTestRunner: all checks pass.OpenRouter server-search boundary
Pines now maps provider-neutral automatic/required web search to OpenRouter's beta
openrouter:web_searchserver tool, including explicit engine selection, bounded result/domain/location policy, fail-closed external-web access, public-URL citation ingestion, request-count accounting, source presentation, and hosted-tool timeline entries. The request, parser, persistence, settings, and UI contracts are covered locally; a live provider call still requires a user-configured OpenRouter key, incurs provider charges, and remains model/route dependent.Real-device evidence and product boundary
A small real-model comparison completed on an iPhone 15 Pro Max with the installed
mlx-community/Qwen3.5-0.8B-MLX-4bitmodel at 4K context, 8 output tokens, and two repeats: 1/1 scenario passed; FP16 median decode was 42.279 tok/s; affine K8/V4 median decode was 42.373 tok/s; the quality gate passed with top-1 agreement 1.0; and no raw fallback was used.That installed Pines binary used an older immutable MLX compatibility pair, not this branch's current
bcf93af+aeaa8e3pair. The result is therefore useful historical runtime smoke only: it does not satisfy current-pair acceptance, import a product evidence tuple, or justify any Verified/Certified, native performance parity, snapshot-restore promotion, or full A16-through-A19 Pro acceptance claim. Exact-current-pair native backend, full benchmark matrix, quality, memory, fallback, and accepted physical-device evidence remain release gates.