Epic: Production Hardening: Wiii Runtime Reliability

[meiiie]

Status

Active planning epic for the post-NVIDIA provider foundation.

Context

Wiii needs to move from demo-capable to production-reliable quickly. PR #169 establishes the NVIDIA provider foundation and local smoke gate, but reliability still needs explicit runtime health, fallback, streaming finalization, pipeline simplification, and memory/identity contracts.

The goal is not just to make requests succeed. The goal is that Wiii feels continuous, responsive, and alive: short chat should stay fast, long/tool/RAG work should stream honestly, memory should be visible in behavior, and provider failures should degrade gracefully instead of freezing the frontend.

Principles

• Keep PRs narrow and mergeable. Do not combine provider runtime, architecture docs, and memory refactors in one PR.
• Do not delete LangGraph/history/compat code in a sweep. Mark, isolate, and remove by phase with rollback notes.
• Prefer health-based runtime decisions over static provider assumptions.
• Preserve Vietnamese-first user-facing behavior.
• Every high-risk change needs tests, rollback notes, and PR reviewer focus.

PR 1: Provider Runtime Reliability

Scope:

• Model-level health probe: if NVIDIA Flash times out, mark it degraded and stop selecting it until recovery conditions pass.
• Per-provider/per-model timeout profiles for chat, streaming, structured/router, and probe paths.
• Same-provider fallback: Flash -> Pro or Pro -> Flash according to current health and configured policy.
• SSE timeout/finalization guard so the frontend never remains stuck in “đang suy nghĩ” after backend/provider failure.
• Ensure structured/router path cannot slow ordinary short chat.

Acceptance:

• Unit tests cover model degraded state, timeout profile selection, same-provider fallback order, and recovery behavior.
• Streaming tests cover timeout/error finalization and done/terminal event behavior.
• Local NVIDIA smoke documents which model was selected and why.
• No secrets or .env* changes are committed.

PR 2: Pipeline Simplification Plan

Scope:

• Document current request lifecycle: request -> auth/org -> memory -> router -> agent -> tool/RAG -> stream.
• Mark active runtime paths vs historical compatibility paths.
• Identify remaining LangGraph/history/compat references and classify them as active, compatibility, test-only, doc-only, or deletion candidate.
• Propose phased LangGraph removal without breaking runtime or rollback.

Acceptance:

• Architecture doc includes lifecycle diagram, ownership, risk, and rollback plan.
• Cleanup list links each LangGraph/history reference to a proposed phase.
• No large runtime deletion in this planning PR unless separately proven safe.

PR 3: Memory/Wiii Identity Reliability

Scope:

• Define the memory contract that prevents failures like “Wiii không nhớ mình”.
• Separate memory namespaces: persona, human, relationship, goals, craft, and world.
• Make living memory behavior visible in chat responses without dumping raw memory.
• Add tests for memory retrieval/injection, relationship continuity, and fallback behavior when memory is unavailable.

Acceptance:

• Memory contract doc and tests prove Wiii can recall stable identity/relationship facts safely.
• Wiii remains clearly AI, not human-impersonating, while feeling continuous and companion-like.
• Memory writes are selective and interpretable, not raw-turn dumps.
• UI-visible behavior explains memory uncertainty gracefully instead of claiming total amnesia.

Risks

• Provider reliability touches high-risk runtime routing and streaming contracts.
• Memory reliability touches privacy, persistence, and user trust.
• LangGraph cleanup can remove hidden compatibility paths if done too aggressively.

Initial Verification Targets

Backend:

cd maritime-ai-service
set PYTHONIOENCODING=utf-8 && pytest tests/unit/ -p no:capture --tb=short -q
ruff check app/ --select=E9,F63,F7

Desktop/streaming when frontend paths change:

cd wiii-desktop
npx vitest run
npx tsc --noEmit
npm run build:embed

Repository hygiene:

git diff --check
git status --short

Related Work

• Builds on PR Add NVIDIA runtime provider and local demo smoke gate #169: NVIDIA runtime provider and local demo smoke gate.
• Related local demo issue: [Ops]: Add local demo login and runtime smoke gate #168.

[meiiie]

Child issues opened for the planned PR sequence:

• Provider Runtime Reliability: health, timeouts, fallback, SSE finalization #171 Provider Runtime Reliability: health, timeouts, fallback, SSE finalization
• Pipeline Simplification Plan: lifecycle map and LangGraph removal phases #172 Pipeline Simplification Plan: lifecycle map and LangGraph removal phases
• Memory and Wiii Identity Reliability: living memory contract and tests #173 Memory and Wiii Identity Reliability: living memory contract and tests

Recommended order remains: merge/chốt NVIDIA foundation PR #169, then implement #171 first, then #172, then #173 unless a production incident changes priority.

[meiiie]

Checkpoint: runtime reliability foundation merged

Completed on 2026-04-28:

• Provider Runtime Reliability: health, timeouts, fallback, SSE finalization #171 closed by Harden provider runtime health, fallback, and SSE finalization #174: provider runtime reliability, model health, timeout profiles, same-provider fallback, and SSE finalization guard.
• Pipeline Simplification Plan: lifecycle map and LangGraph removal phases #172 closed by Document Wiii pipeline simplification plan #175: pipeline simplification lifecycle map and phased de-LangGraph removal plan.
• Memory and Wiii Identity Reliability: living memory contract and tests #173 closed by Enforce Wiii memory identity contract #176: Wiii memory identity contract, memory block separation, Direct/Tutor/RAG/Memory prompt propagation, and regression coverage.

Current main checkpoints:

• Harden provider runtime health, fallback, and SSE finalization #174 merged as 6da10e2.
• Enforce Wiii memory identity contract #176 merged as e5c9087.
• Document Wiii pipeline simplification plan #175 merged as ec776e9.
• Local and CI verification on the merged PRs passed required gates; CodeRabbit on Document Wiii pipeline simplification plan #175 was bypassed only for review-rate-limit after human approval and no unresolved threads.

Recommended next focus:

• Start the first implementation slice from the pipeline simplification plan: isolate the runtime loop behind a narrow WiiiRunner interface before deleting remaining LangGraph/history compatibility references.
• Add local demo smoke coverage that exercises NVIDIA provider chat + SSE finalization + memory continuity in one script.
• Then begin memory persistence hardening: prompt-injection audit for stored memory, provenance labels, and UI-visible memory confidence behavior.

[meiiie]

Checkpoint update: PR #178 has been merged.

Completed in this checkpoint:

• Introduced native Wiii runtime contracts for turn requests, run context, turn state, turn result, and stream events.
• Added sync and streaming facades over the existing WiiiRunner adapters without changing runtime behavior.
• Preserved the current process_with_multi_agent and legacy tuple streaming contracts so migration can proceed incrementally.
• Added focused regression coverage for contract adaptation, stream wrapping, and golden parity surfaces.

Verification reported on PR #178:

• python -m pytest tests/unit/test_wiii_runtime_contracts.py tests/unit/test_wiii_runner_golden_parity.py tests/unit/test_runner_node_surface.py -q
• python -m pytest tests/unit/test_chat_request_flow.py tests/unit/test_chat_stream_coordinator.py tests/unit/test_graph_stream_runtime.py -q
• python -m ruff check app/ tests/unit/test_wiii_runtime_contracts.py tests/unit/test_wiii_runner_golden_parity.py --select=E9,F63,F7
• git diff --check

Next recommended checkpoint: route one non-risky entrypoint through run_wiii_turn / stream_wiii_turn, then add a local FE smoke path that proves the chat lifecycle cannot hang on stream finalization.

[meiiie]

Checkpoint update: PR #180 has been merged.

Completed in this checkpoint:

• Sync chat multi-agent execution now enters runtime through WiiiTurnRequest + run_wiii_turn.
• Default SSE V3 multi-agent streaming now enters runtime through stream_wiii_turn.
• WiiiStreamEvent preserves StreamEvent-like shape so existing SSE presenter serialization remains unchanged.
• Injected native one-argument stream functions are supported explicitly via signature detection.
• CodeRabbit review threads were addressed and resolved.

GitHub governance checkpoint:

• Issue Ops: reconcile main branch admin override drift #181 reconciled branch-protection drift back to the documented admin override policy: enforce_admins=false.
• Required reviews, CODEOWNERS review, last-push approval, required Gate Summary, conversation resolution, and linear history remain configured for non-admins.

PR #180 CI result before merge:

• Gate Summary: success
• Backend test: success
• CI Unit Tests: success
• CI Docker Build: success
• Build Images: success
• Lint: success
• Security Audit: success
• CodeQL Python/JS: success
• CodeRabbit status: failure/rate-limited, but all review threads were addressed/resolved.

Next recommended checkpoint: move one more call site outside /chat (for example scheduled tasks or webhook adapters) onto WiiiTurnRequest, then add a local runtime smoke that exercises provider selection + stream finalization end-to-end.