Skip to content

roadmap.md

Latest commit

 

History

History
168 lines (124 loc) · 4.91 KB

roadmap.md

File metadata and controls

168 lines (124 loc) · 4.91 KB
Raw

Delivery Roadmap

The roadmap is evidence-gated. Later phases are not started merely because they are technically interesting.

Phase 0 — Repository and contract foundation

Deliverables:

  • focused README and documentation index;
  • normative CLI MVP contract;
  • package and dependency definition;
  • source and test layout;
  • formatting, linting, typing, and test commands;
  • GitHub Actions quality gates;
  • fixture provenance, license, and data-handling policy.

Exit conditions:

  • a clean clone can install the project and run CLI help and tests through one documented flow;
  • command, input, output, exit-code, authentication, and fixture contracts are versioned;
  • unsupported inputs and untrusted-data boundaries are explicit.

Phase 1 — Offline and local rule-based triage

Deliverables:

  • fixture and local-file input modes;
  • manifest and evidence schemas;
  • input validation and path safety;
  • secret redaction;
  • failure-window extraction;
  • deterministic category rules;
  • unknown and abstention;
  • terminal, Markdown, and JSON reports;
  • fixture, golden, and adversarial tests.

Exit conditions:

  • all supported fixtures produce expected redacted evidence and safe decisions;
  • no configured secret survives redaction tests;
  • unsupported logs fail safely;
  • deterministic rule scores cannot be confused with calibrated probabilities;
  • maintainers can review downloaded historical logs without GitHub credentials.

Phase 2 — Read-only GitHub collection

Deliverables:

  • environment-based token authentication;
  • least-privilege permission documentation;
  • workflow-run and failed-job selection;
  • bounded log and annotation download;
  • normalized local input bundles;
  • rate-limit, retry, and authentication errors;
  • mocked GitHub API tests.

Exit conditions:

  • public fixture repositories work end to end;
  • private repository access requires only documented read permissions;
  • tokens never appear in logs, files, or reports;
  • offline behavior remains fully usable without network access.

Phase 3 — User and taxonomy validation

Deliverables:

  • maintainer interview notes;
  • reviewed historical failures;
  • cause-category and temporal-assessment guides;
  • inter-reviewer agreement report;
  • evidence-usefulness and rule-coverage reports;
  • build, simplify, pivot, or stop decision.

Exit conditions:

  • at least three maintainers provide repeated reviewed failures;
  • at least two repeat the workflow or express read-only beta intent;
  • the taxonomy reaches acceptable agreement or is simplified;
  • evidence extraction is useful during abstention;
  • nontrivial cases remain beyond deterministic rules.

Phase 4 — Reproducible ML baseline

Deliverables:

  • append-only label ledger;
  • immutable snapshot manifest;
  • repository-grouped and temporal splits;
  • TF-IDF plus metadata baseline;
  • calibration and selective prediction;
  • leakage tests and error analysis;
  • model card;
  • comparison with rule-only behavior.

Exit conditions:

  • the model improves reviewed natural-data performance at useful coverage;
  • risky categories meet precision policy;
  • evaluation is reproducible from a clean environment;
  • no known leakage or schema-compatibility check fails.

Phase 5 — GitHub Actions service vertical slice

Deliverables:

  • least-privilege GitHub App;
  • webhook signature verification;
  • idempotent event handling;
  • bounded job-log ingestion;
  • GitHub Check summary;
  • explicit correction action;
  • rerun reconciliation;
  • retention and deletion controls.

Exit conditions:

  • one fixture repository works end to end;
  • at least one consenting beta repository receives repeated useful results;
  • private logs do not appear in telemetry or public artifacts;
  • failure, retry, deletion, and incident behavior are documented.

Phase 6 — Controlled model operations

Deliverables:

  • candidate and champion references;
  • shadow comparison;
  • delayed-label report;
  • manual promotion gate;
  • application and model rollback procedure;
  • product, model, and system metrics;
  • one published failure drill and incident report.

Exit conditions:

  • a bad candidate is prevented from affecting users;
  • an intentionally bad release can be rolled back through the documented procedure;
  • dashboards and alerts correspond to actual operator actions.

Phase 7 — Beta evidence

Deliverables:

  • 3–5 beta repositories;
  • processed-failure and feedback summary;
  • useful-triage rate;
  • time-to-next-action comparison;
  • blind-rerun analysis;
  • cost, privacy, and reliability report;
  • revised product decision.

Scope expansion

Add another test framework, language, CI provider, advanced retrieval method, or deployment platform only when beta evidence identifies it as the next highest-value constraint.

Never schedule by default

  • Kubernetes;
  • Kafka;
  • feature store;
  • distributed training;
  • LLM root-cause generation;
  • automatic remediation;
  • multiple frontends;
  • multi-region infrastructure.

These require measured demand, not portfolio aesthetics.