Technical Design Document (TDD): Sovereign Renewal of oxidizedGraph (Rust Orchestrator)

[stevei101]

{"response": "Technical Design Document (TDD): Sovereign Renewal of oxidizedGraph (Rust Orchestrator). Status: Strategic Planning & Implementation Ready. Core Stack: Rust (tokio, axum, kube-rs), A2A Protocol, Azure AKS, AWS EKS, Crossplane, dockworker.ai. --- EXECUTIVE SUMMARY: To aggressively pursue our 'AI First, Hire Later' strategy and maintain the 50-70% OpEx reduction, it is time to renew and harden 'oxidizedGraph' in the stevedores-org repository. This Rust-based DAG orchestrator is the memory-safe 'Sovereign Brain' of Lornu AI. It replaces human project managers by scheduling, spawning, and reaping ephemeral ADK Python agents (uv-managed) as Kubernetes Jobs. This renewal focuses on high-concurrency state management, K8s API integration, and investor-ready audit logging. --- EPIC 1: High-Concurrency DAG State Engine (Rust Core). Strategic Goal: Harden the Tokio-based asynchronous engine to process thousands of simultaneous Agent-to-Agent (A2A) transitions without race conditions or memory leaks. ROI Lens: Eliminates the need for expensive third-party orchestration SaaS, reducing core infrastructure OpEx by 40%. FEATURE 1.1: Lock-Free State Machine Maturation. USER STORY: As the Principal Architect, I want oxidizedGraph to utilize a lock-free, Rust-native state machine, so that 'Feature Briefs' transition from 'Scoping' to 'Implementation' seamlessly across distributed agent workers. ACCEPTANCE CRITERIA: The engine must use 'tokio' for non-blocking task execution. Agent states must be synchronized with the Sovereign Knowledge Fabric (Firestore/CosmosDB). Must pass strict Rust 'clippy' and memory-safety CI checks. TECHNICAL TASKS: 1. [Rust] Refactor the 'StateEngine' struct in src/dag/ to utilize thread-safe crossbeam channels. 2. [Backend] Expose an Axum-based JSON-RPC 2.0 endpoint for the A2A bus. 3. [Logistics] Ensure the dockworker.ai multi-stage build creates a highly optimized, distroless OCI image for the Rust binary. --- EPIC 2: Ephemeral Compute Orchestration (K8s API Integration). Strategic Goal: Empower oxidizedGraph to directly interface with the Kubernetes API to dynamically spawn and destroy ADK worker agents. ROI Lens: Maximizes Azure AKS/AWS EKS Spot Instance utilization, ensuring compute costs are exactly aligned with active agent work (Scale-to-Zero). FEATURE 2.1: Autonomous Job Provisioning and Reaping. USER STORY: As an SRE Agent, I want oxidizedGraph to spawn 'uv'-managed Python ADK agents as ephemeral v1.Job resources, so that we do not pay for idle container execution. ACCEPTANCE CRITERIA: Must utilize the 'kube-rs' crate to authenticate via Azure AD Workload Identity (or AWS IRSA). Automatically inject required Crossplane references and the 'lornu-ai-final-clear-bg.png' metadata into the generated Job manifests. Must include a 'Reaper' loop that deletes completed/failed Jobs after exporting telemetry. TECHNICAL TASKS: 1. [Rust] Implement the 'KubeClient' module using 'kube-rs'. 2. [K8s] Define the Base Kustomize template for the ephemeral ADK Job. 3. [Security] Provision a scoped ClusterRole ensuring the orchestrator can only manage specific 'lornu-agent' namespaces. --- EPIC 3: Sovereign Audit & Telemetry (Investor Readiness). Strategic Goal: Stream all DAG orchestration events to the Data Fabric, proving the autonomous system's efficiency and reducing human intervention to zero. ROI Lens: Provides tangible, cryptographically verifiable proof of the 50-70% engineering efficiency gains for equity funding due diligence. FEATURE 3.1: Immutable A2A Event Ledger. USER STORY: As the CFO Agent, I want oxidizedGraph to log every agent deployment, execution time, and task outcome, so that I can generate automated OpEx and efficiency reports. ACCEPTANCE CRITERIA: All A2A handoffs must emit an OpenTelemetry (OTel) trace. Metrics must be aggregated and pushed to the Azure Hub / GCP BigQuery instance. TECHNICAL TASKS: 1. [Rust] Integrate 'tracing' and 'opentelemetry' crates into the axum server. 2. [Data] Define the 'OrchestrationLedger' schema for the Data Fabric. --- EPIC 4: Governance and Documentation Handshake. Strategic Goal: Ensure the renewed oxidizedGraph strictly adheres to the Lornu AI 6-File Rule before reaching production. FEATURE 4.1: Autonomous Repository Alignment. USER STORY: As the Librarian Agent, I want oxidizedGraph to maintain perfect documentation, so that human contributors and other agents understand the DAG orchestration logic. ACCEPTANCE CRITERIA: The repository must pass the 6-File Audit (.cursorrules, AGENTS.md, CLAUDE.md, README.md, .github/copilot-instructions.md, .github/system-instruction.md). Must clearly document the 'No Dockerfile' OCI build process via dockworker.ai. TECHNICAL TASKS: 1. [Librarian] Audit and update 'stevedores-org/oxidizedgraph/AGENTS.md' to define the Orchestrator persona. 2. [Governance] Add Kyverno policies to ensure oxidizedGraph pods cannot run without proper OIDC/WIF annotations."}