Skip to content

fix(scripts,docs): make a fresh clone actually work before going public - #49

Merged
wms2537 merged 2 commits into
mainfrom
fix/production-readiness-pass
Aug 1, 2026
Merged

fix(scripts,docs): make a fresh clone actually work before going public#49
wms2537 merged 2 commits into
mainfrom
fix/production-readiness-pass

Conversation

@wms2537

@wms2537 wms2537 commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

A production-readiness pass run against a fresh clone of this repo — i.e. what an arriving contributor actually gets, not what works on a machine that already has everything.

The engineering gates were already green. What was broken was everything around the code: the entry path, the harnesses, and the story the docs tell.

Blockers a first-time visitor would hit

The from-source quickstart started nothing. scripts/run_stack.sh — the last step of both the README quickstart and the CONTRIBUTING dev setup — resolved every path from a hardcoded /home/soh/cctv. With no precondition checks it failed silently, printing stack up: mediamtx=… core=… vite=… and then sleeping for 30 minutes. Reproduced verbatim:

mkdir: /home/soh: Operation not supported
run_stack.sh: line 17: /home/soh/cctv/data/stack_logs/mediamtx.log: No such file or directory
stack up: mediamtx=57008 core=57013 vite=57014 (auto-stop in 1800s)

Now repo-relative, honours HELDAR_DATA_DIR, and exits non-zero on a missing core binary / MediaMTX / dashboard deps. The same hardcoded root is gone from smoke_web.sh and all eight validate_*.sh.

setup_mediamtx.sh only worked on Linux/x86. It parsed the release tag with GNU-only grep -oP and always downloaded linux_amd64 — so it failed on a dev Mac and on the arm64 appliance target. Now detects OS/arch (linux + darwin; amd64/arm64/armv7/armv6) and parses the tag portably.

The harnesses were not validating anything

Publish authorization moved to the kernel (authMethod: http/internal/mediamtx-auth), which means an ffmpeg publisher started before heldar-core gets a 401 Unauthorized and exits immediately. validate.sh, smoke_web.sh and the Playwright e2e_stack.sh all started publishers first — so every one of them was exercising cameras that never streamed.

Before / after on validate.sh, same clone:

before after
camera state offline, 5 reconnects recording
segments 0 7
fps / bitrate null 15.0 / 358 kbps
clip cut 633 KB mp4, 14 s, no gaps

e2e_stack.sh had a second, independent bug: it runs the core on :8011 for isolation, but mediamtx.yml pins the auth callback to :8000, so MediaMTX asked a dead port and denied every publish and read. It now starts MediaMTX from a port-adjusted copy of the config. Both e2e cameras reach recording, 8 segments, 10 fps.

Two macOS portability fixes there as well: wait "${PIDS[-1]}" needs bash ≥ 4.3 and aborted under set -u on macOS's bash 3.2, and fuser -k has no -k on macOS (falls back to lsof).

Docs still described the retired generated-tree model

LICENSING.md, DESIGN-PRINCIPLES.md #8, the open-core and module-system pages (plus es/zh-Hans), REMOTE-ACCESS.md, PRODUCTION.md, and the heldar-server composition-root comments all said the public repo is generated from a private monorepo and that main.rs is substituted per build.

That directly contradicts CONTRIBUTING (which correctly says there is no mirror and no import step) and would tell an arriving contributor their PR gets regenerated away. Rewritten to the real model: this repo is the source of truth, and a private product composes its own binary against heldar_server::run(impl Verticals). ARCHITECTURE.md §21 also no longer points at a docs/adr/ path that does not exist here.

Verified on the fresh clone

cargo fmt --check · clippy --workspace --all-targets -D warnings · 288 tests · read-seam lint · open build (--no-default-features, no wasmi linked) · npm run typecheck && npm run build · Docusaurus build — all pass. setup_mediamtx.sh verified on darwin/arm64, where the old one failed twice.

Also checked and clean, no change needed: no secrets in the full 234-commit history; ghcr images anonymously pullable (0.3.0/0.3.1/latest); the advertised install.sh is live and byte-identical to website/static/install.sh, and the raw compose URLs it fetches return 200; crate metadata complete and consistent at 0.3.1; relative markdown links resolve.

Not in this PR — needs a decision

  • SECURITY.md points at a disabled feature. It directs reporters to GitHub Private Vulnerability Reporting, but that is {"enabled": false} on this repo — so the documented channel dead-ends and a researcher's only remaining option is the public issue the policy forbids.
  • No branch protection or ruleset on main — CI is advisory and force-push is possible.
  • docs/adr/ is absent while shipped code and docs cite ADR 0001/0003/0004.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

wms2537 added 2 commits July 31, 2026 15:06
Findings from a production-readiness pass done against a fresh clone of this
repo, i.e. what an arriving contributor actually gets.

Scripts (the entry path both README and CONTRIBUTING point at):
- run_stack.sh resolved paths from a hardcoded /home/soh/cctv and had no
  precondition checks, so elsewhere it started nothing yet still printed
  'stack up: ...' and slept 1800s. Repo-relative now, honours HELDAR_DATA_DIR,
  and fails loudly on a missing core binary / MediaMTX / dashboard deps. Same
  hardcoded root removed from smoke_web.sh and every validate_*.sh; CAM= now
  overrides the camera they exercise.
- setup_mediamtx.sh used GNU-only 'grep -oP' and always fetched linux_amd64;
  it now detects OS/arch and parses the tag portably (MEDIAMTX_TAG= pins one).
- Publish authorization moved to the kernel (authMethod: http ->
  /internal/mediamtx-auth), so an ffmpeg publisher started before heldar-core
  gets a 401 and exits. validate.sh, smoke_web.sh and e2e_stack.sh all started
  publishers first, so they were validating cameras that never streamed. They
  now publish after the API is healthy; validate.sh aborts if its camera dies.
- e2e_stack.sh additionally ran the core on :8011 while mediamtx.yml pins the
  auth callback to :8000, so MediaMTX denied every publish and read: it now
  starts MediaMTX from a port-adjusted copy. Plus two macOS portability fixes
  (bash 3.2 has no negative array subscripts; fuser has no -k).

Docs: LICENSING, DESIGN-PRINCIPLES #8, the open-core and module-system pages
(with es/zh-Hans), REMOTE-ACCESS, PRODUCTION, README and the heldar-server
composition-root comments still described the retired generated-tree model --
a public repo generated from a private monorepo, with main.rs substituted per
build. That directly contradicted CONTRIBUTING and would tell a contributor
their PR gets regenerated away. They now describe the real model, and
ARCHITECTURE.md no longer points at a docs/adr/ path absent from this repo.

Verified on a fresh clone: fmt, clippy -D warnings, 288 tests, read-seam lint,
open build and the dashboard build all pass; validate.sh records 7 segments at
15fps and cuts a clip; the e2e stack brings both cameras to 'recording'.

Signed-off-by: wms2537 <51080539+wms2537@users.noreply.github.com>
Shipped code and docs cited ADR 0001/0003/0004 as the design records of the
open platform, but docs/adr/ did not exist here -- contributors were pointed
at documents they could not read.

Publishes the two that describe THIS codebase:
- 0003: remote viewing over WebRTC; retiring the mobile app + kernel-managed
  WireGuard. Trimmed before publishing: the phase-by-phase operational log
  (infrastructure secret names, deployment TODOs) and the list of deliberately
  deferred security hardenings on the internet-facing relay -- an attacker
  roadmap that does not belong in public. The decision, rationale, alternatives,
  consequences and risks are intact.
- 0004: edge nodes stay on SQLite; scale by adding nodes rather than porting to
  Postgres. Cross-references rewritten: it cited 'issue #36' (a dependabot PR in
  this repo), CLAUDE.md (a private file), and ADR 0001 (unpublished).

Decisions wholly about the commercial tier stay private; docs/adr/README.md
says so rather than leaving the gap unexplained.

Also: services/embeddings.rs credited the no-vector-DB/no-ANN choice to ADR
0004, which is the SQLite-vs-Postgres brief -- no ADR records that decision, so
the rationale now stands on its own. CONTRIBUTING notes that 'issue #NN' in
comments predating 2026-07 refers to the pre-flip tracker, not this repo (65
such references, where e.g. #38 is now a setup-node bump and #46 is a live
good-first-issue).

Signed-off-by: wms2537 <51080539+wms2537@users.noreply.github.com>
@wms2537
wms2537 merged commit 34af9ae into main Aug 1, 2026
5 checks passed
@wms2537
wms2537 deleted the fix/production-readiness-pass branch August 1, 2026 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant