Skip to content

feat: add a performance dashboard tying latency benchmarks (G1–G4) to a live chart #53

Description

@sr-857

H3. feat: add a performance dashboard tying latency benchmarks (G1–G4) to a live chart

Labels: visualization, performance

Priority: Low

Proposed Solution: Extend pipeline_dashboard.html to plot the CI-tracked benchmark history from B2/G4 rather than a static number.

Acceptance Criteria: [ ] dashboard reads from committed benchmark artifacts

Dependencies: B2, G4

Estimated Difficulty: M

Estimated Time: 1 day


16. Milestones / Project Board

Milestone Scope Key Issues
M1 — Correctness Hardening Fix the real bugs before anything else A1, A2, A4, A6, A7, A8
M2 — Validation Prove the numbers are right, not just present A3, A5, D2
M3 — CI/Test Breadth Cover CUDA, ML, closed-loop, fuzz, sanitizers B1–B7
M4 — Architecture & Hygiene Cleanup Build system, repo bloat, formatting C1–C6, F1–F6
M5 — Performance Proof Real end-to-end numbers, SIMD, CUDA comparison G1–G4
M6 — Submission Polish Docs, visualizations, ML reproducibility, assumptions stated D1, D3, D4, E1–E5, H1–H3

17. Development Roadmap

**Immediate (this week): ** A1, A2, A4, A6 — these are the correctness bugs a judge or careful reviewer would find fastest, and they're each ≤2 days.

**Next week: ** A3, A5, A8, B3, B5, B6 — validation tests and sanitizer/fuzz coverage, so the "it works" claim has evidence behind it.

**Next month: ** B1, B2, B4, D1, D2, C1, C2, F1–F4 — CI breadth, ML reproducibility, build consolidation, GitHub hygiene.

**Pre-submission: ** G1–G4 (real end-to-end performance numbers, since "10 ms real-time" is a headline PS9 criterion and needs to survive scrutiny), E5 (dataset assumptions stated), C6 (assumptions documented).

**Post-submission / research publication track: ** D3, D4, E3, H1–H3 — the kind of polish that matters more for an SN Computer Science-style paper than for the hackathon deadline itself.


18. Final Scorecard (0–10)

Category Score Rationale
Scientific Correctness 7 Core math is standard and appears implemented correctly on inspection; unproven by dedicated golden-value/end-to-end validation tests
Software Engineering 6 Real modular C library with a working build, but struct-duplication landmine (A2) and 9 redundant build scripts (C1)
Documentation 8 Unusually thorough for a hackathon repo — 11 docs files, dedicated compliance mapping doc, paper draft
Mathematics 7 Zernike/Fried/Kolmogorov/coupling models are textbook-appropriate; missing convergence/condition reporting (A8)
AI/ML 6 Mature pipeline breadth (MLP/CNN/LSTM/ablation/noise-robustness), but headline metrics not reproducible from a fresh clone (D1/B4)
Performance 6 OpenMP + CUDA present; latency claims not independently, end-to-end verified (G1)
Real-Time Capability 6 Architecturally sound for <10 ms, but the actual end-to-end number (vs. hot-path-only microbenchmark) is unverified
Visualization 8 Strong breadth of static and interactive visualizations
Research Depth 7 Correct concepts, missing explicit literature citation trail (E3)
Innovation 6 Predictive-AO LSTM and coupling-based DM model are reasonable, not groundbreaking, choices — appropriate for the problem
Reproducibility 5 Weakest category — ML artifacts and some benchmark numbers cannot currently be regenerated from git clone alone
Maintainability 6 Good in-code documentation and naming; undermined by struct duplication (A2) and build-script sprawl (C1)

Composite: 6.8 / 10


19. Risk Assessment

Risk Likelihood Impact Mitigation
ISRO evaluators ask to reproduce the ML/latency numbers and can't from the public repo High High D1, B4, G1
A hidden test-set-vs-real-dataset geometry mismatch surfaces only at submission time Medium High A7, E5
Struct-duplication bug (A2) causes a crash under a future contributor's change, discovered late Medium High A2, address before any further feature work on rippra_api.c
Reviewer discounts the submission for unlabeled simplifying assumptions (DM coupling model) looking like an oversight rather than a deliberate choice Medium Medium C6, E5
Undiscovered memory bug in the C core surfaces during a live demo Low–Medium High B6 (sanitizers), B5 (fuzz)

20. Final Verdict

🟢 Submit-capable with a focused hardening pass, not a rewrite. The prior stale audit's "do not submit" verdict no longer reflects the codebase. The core PS9 deliverables are implemented with reasonable algorithmic choices. The highest-leverage work between now and submission is not new features — it's (1) fixing the handful of real correctness bugs in Category A, (2) proving the numbers with validation tests and end-to-end benchmarks rather than asserting them in docs, and (3) making the ML claims reproducible from a clean clone. Everything in Categories C, F, and H is polish that helps a reviewer trust the project but won't change whether the core algorithm works.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions