[BE-2] Audit Log + Merkle Tree, LLM Guard Enhancements & Missing API Endpoints

[iamyxsh]

[BE-2] Audit Log + Merkle Tree, LLM Guard Enhancements & Missing API Endpoints

Labels: backend, priority:high, week-1-4
Assignee: Backend Dev (audit log), Yash (Merkle tree)

---

Context

Per Sections 7, 5.1, and 17 of the Source of Truth, every decision (approve/deny) must be logged in a tamper-evident audit log with an incremental Merkle tree. The current system has alerts and spend records but lacks the formal audit trail. Additionally, several planned LLM guard features and API endpoints are not yet implemented.

---

1. Audit Log System (Section 7 — Week 1-2)

Files: src/audit/

• AuditEntry struct per Section 18:

  struct AuditEntry {
      id: u64,                        // auto-increment
      timestamp: u64,                 // unix timestamp (ms)
      intent_type: String,            // "api_call" | "onchain_tx"
      service: String,                // "openai" | "binance" | "0xUniswap..."
      action: String,                 // endpoint path or function selector
      decision: String,               // "approved" | "denied"
      reason: Option<String>,         // null if approved, reason if denied
      cost_usd: Option<f64>,          // estimated cost
      policy_version_hash: H256,      // hash of policy at time of decision
      intent_hash: H256,              // hash of the full intent
      permit_hash: Option<H256>,      // hash of signed permit (onchain + approved)
      merkle_root: H256,              // tree root AFTER this entry appended
  }

• SQLite audit_log table — indexed on timestamp, service, decision
• Log every proxy decision — both approved and denied requests, across all proxy types (OpenAI, Anthropic, Binance, generic, onchain)
• Intent hashing — keccak256 hash of the full intent for each entry
• Policy version hash — hash of active fishnet.toml at time of decision (from config hot-reload)

2. Incremental Merkle Tree (Section 7.3)

Owner: Yash | Files: src/audit/merkle.rs

• Binary Merkle tree with Keccak-256 hash function (tiny-keccak crate)
• Append-only — entries are never modified or deleted
• Store leaf hashes + intermediate nodes in a separate SQLite table
• Merkle root stored in each AuditEntry — root computed after appending the entry
• Tamper-evidence — if any historical entry is modified, the root diverges
• Foundation for ZK proofs — tree must support Merkle path extraction for proof generation (Section 12)

3. LLM Guard Enhancements (Section 5.1)

• Model restrictions:
  • Parse model field from request body JSON
  • Check against allowed_models list in policy
  • Deny with reason "model not in allowlist: {model}" if not permitted
• Token usage parsing from response:
  • OpenAI: parse usage.total_tokens, usage.prompt_tokens, usage.completion_tokens from response body
  • Anthropic: parse usage.input_tokens, usage.output_tokens from response body
  • Compute actual cost using model pricing table (map model name -> cost per 1K tokens)
  • Update daily spend counter with actual cost (not just estimated)
• Pricing table:
  • gpt-4o: $2.50/$10 per 1M input/output tokens
  • gpt-4o-mini: $0.15/$0.60 per 1M input/output tokens
  • claude-sonnet: $3/$15 per 1M input/output tokens
  • (configurable/extensible for new models)

4. Missing API Endpoints (Section 17)

These endpoints are consumed by the dashboard and are not yet implemented:

• GET /api/status — {running: true, uptime: "2h 34m", services: ["openai", "binance"], today_spend: {openai: 12.50, binance: 0}, today_requests: {openai: 342, binance: 15}}
• GET /api/policies — return parsed fishnet.toml as JSON object
• PUT /api/policies — accept updated policy JSON, write back to fishnet.toml, trigger hot-reload, return {saved: true, policy_hash: "0x..."}
• GET /api/audit?from=&to=&service=&decision=&page= — paginated audit log query, returns {entries: [...], total: 1247, page: 1, pages: 63}
• GET /api/audit/export — return CSV file download of audit log (with Content-Disposition header)
• GET /api/spend?days=30 — {daily: [{date: "2026-03-01", service: "openai", amount: 15.20}, ...]}

---

Acceptance Criteria

• Every proxy decision (approve or deny) creates an AuditEntry with all fields populated
• Merkle tree root is updated and stored on every append
• Modifying any historical audit entry causes root divergence (tamper-evidence)
• Model restrictions block unauthorized models with clear error messages
• Token usage is parsed from actual responses and spend counters reflect real costs
• All 6 API endpoints respond correctly and are consumed by the dashboard