feat: Update execution plans with external capability verification and refine embedding provider integration

Author: devlux76

CORTEX-DESIGN-PLAN-TODO.md

@@ -33,10 +33,15 @@ Completed since the prior snapshot:
 6. Deterministic dummy SHA-256 embedder exists for pre-model hotpath testing (`embeddings/DeterministicDummyEmbeddingBackend.ts`).
 7. Benchmark harness exists for dummy embedder throughput baselining (`npm run benchmark:dummy`).
 8. Baseline adaptive provider selection exists with capability filtering + benchmark-based winner choice (`embeddings/ProviderResolver.ts`, `embeddings/EmbeddingRunner.ts`).
+9. External capability verification completed for real-provider planning:
+	- Transformers.js is ONNX Runtime-backed and directly exposes `webnn`, `webgpu`, and `wasm` device paths.
+	- Transformers.js does not currently expose `webgl` as a direct device; `webgl` should remain an explicit ORT adapter path.
 
 Next focus:
 1. Wire resolved model profiles into runtime ingest/query entry points.
-2. Add real embedding providers (ONNX/Transformers/WebNN/WebGPU/WebGL/WASM) to resolver candidate sets.
+2. Add real embedding providers to resolver candidate sets, split by runtime family:
+	- Transformers.js provider (`webnn/webgpu/wasm`)
+	- Explicit ORT WebGL provider (`webgl`)
 3. Add browser/electron runtime test lanes to match merge-gate policy.
 
 ## 1. Design
@@ -150,18 +155,23 @@ Performance budget targets for v1:
 
 Graceful degradation:
 1. `webgpu` preferred.
-2. `webgl` fallback.
-3. `webnn` optional path for matmul-friendly ops.
+2. `webnn` optional path for matmul-friendly ops.
+3. `webgl` fallback via explicit ORT adapter path.
 4. `wasm` guaranteed baseline.
 
+Implementation note (verified 2026-03-11):
+1. Transformers.js path currently maps to `webnn/webgpu/wasm` (no direct `webgl` device key).
+2. Keep `webgl` in architecture through the explicit ORT adapter backend.
+
 ### 1.9 Current gap analysis from repo snapshot
 Observed blockers in current PoC files:
-1. Embedding runtime modules (provider resolver + runner) are still missing.
-2. Ingest/query orchestrators are not yet wired to resolved `ModelProfile` values.
-3. Browser/Electron runtime test lanes are not yet implemented in scripts/CI.
-4. Shader and backend files compile but are not yet integrated into a full vertical runtime path.
+1. Embedding runtime modules exist (`ProviderResolver` + `EmbeddingRunner`), but only baseline/dummy-provider flow is wired.
+2. Real provider adapters are not yet wired (Transformers.js for `webnn/webgpu/wasm`; explicit ORT adapter for `webgl`).
+3. Ingest/query orchestrators are not yet wired to resolved `ModelProfile` values.
+4. Browser/Electron runtime test lanes are not yet implemented in scripts/CI.
+5. Shader and backend files compile but are not yet integrated into a full vertical runtime path.
 
-These are Phase 0 blockers and should be fixed before feature work.
+These are the remaining vertical-slice blockers before broader feature expansion.
 
 ## 2. Implementation Plan
 
@@ -318,7 +328,7 @@ Priority legend:
 6. Add corruption recovery tooling for vector store and metadata store.
 7. Add schema migration tests across multiple versions.
 8. Add large-corpus stress tests for memory and latency.
-9. Add adaptive runtime policy based on backend capability.
+9. Extend adaptive runtime policy with real providers and runtime telemetry persistence.
 10. Add resource governance controls for Daydreamer CPU budget.
 11. Improve ranking quality with optional rerank stage.
 12. Add developer docs with architecture diagrams and troubleshooting.

PROJECT-EXECUTION-PLAN.md

@@ -38,10 +38,27 @@ Completed in this pass:
    - selection now supports capability filtering + benchmark-based winner choice
 
 Open items carried to next pass:
-1. Wire resolved `ModelProfile` into first concrete ingest/query orchestrator path (once those runtime modules are added).
+1. Wire resolved `ModelProfile` into first concrete ingest/query orchestrator path.
 2. Add real embedding providers (ONNX/Transformers/WebNN/WebGPU/WebGL/WASM) as candidates for the resolver.
 3. Add browser/electron runtime scripts and CI lanes for non-Node merge gating.
 
+### External Capability Verification (2026-03-11)
+
+Verified during this pass (hard handoff facts):
+1. Transformers.js is ONNX Runtime-backed (docs + upstream source).
+2. Transformers.js device mapping exposes `webnn`, `webnn-gpu`, `webnn-cpu`, `webnn-npu`, `webgpu`, and `wasm` for web environments.
+3. Transformers.js does not currently expose `webgl` as a direct `device` type; WebGL should remain an explicit ORT adapter path.
+4. Node-side ONNX providers (platform-dependent) include `cuda`, `dml`, and `coreml` in upstream mapping.
+
+Source anchors to re-check quickly in a new session:
+1. `huggingface/transformers.js` -> `packages/transformers/src/backends/onnx.js`:
+   - `DEVICE_TO_EXECUTION_PROVIDER_MAPPING`
+   - `deviceToExecutionProviders(...)`
+2. `huggingface/transformers.js` -> `packages/transformers/src/utils/devices.js`:
+   - `DEVICE_TYPES`
+3. Transformers.js docs (`v3.8.1` and `main`):
+   - index + WebGPU guide + ONNX backend API pages
+
 ## Next Session Highest Priority (P0)
 
 Connect adaptive embedding selection to runtime orchestration and add real provider candidates.
@@ -61,7 +78,9 @@ Definition of done for this pass:
 
 1. Strict TDD: Red -> Green -> Refactor for every slice.
 2. Runtime realism: browser and Electron lanes are required merge gates.
-3. Provider fallback policy: `webnn -> webgpu -> webgl -> wasm`.
+3. Provider fallback policy:
+   - Transformers.js path: `webnn -> webgpu -> wasm`
+   - Explicit ORT path: `webnn -> webgpu -> webgl -> wasm`
 4. Numeric ownership: model-derived values from profile; policy values from declared policy objects.
 
 ## Execution Sequence
@@ -76,8 +95,8 @@ Definition of done for this pass:
 5. Implement embedding runner with fallback chain and telemetry:
    - `embeddings/EmbeddingRunner.ts` ✅ baseline done (2026-03-11)
    - `embeddings/ProviderResolver.ts` ✅ baseline done (2026-03-11)
-   - `embeddings/TransformersEmbeddingBackend.ts`
-   - `embeddings/OrtWebglEmbeddingBackend.ts`
+   - `embeddings/TransformersEmbeddingBackend.ts` (targeting `webnn/webgpu/wasm`)
+   - `embeddings/OrtWebglEmbeddingBackend.ts` (explicit `webgl` path)
    - `embeddings/OnnxEmbeddingRunner.ts`
 6. Build Hippocampus ingest using profile-derived chunking/dimensions.
 7. Build Cortex retrieval using profile-derived routing/truncation policies.