fix(scripts,docs): make a fresh clone actually work before going public - #49
Merged
Conversation
Findings from a production-readiness pass done against a fresh clone of this repo, i.e. what an arriving contributor actually gets. Scripts (the entry path both README and CONTRIBUTING point at): - run_stack.sh resolved paths from a hardcoded /home/soh/cctv and had no precondition checks, so elsewhere it started nothing yet still printed 'stack up: ...' and slept 1800s. Repo-relative now, honours HELDAR_DATA_DIR, and fails loudly on a missing core binary / MediaMTX / dashboard deps. Same hardcoded root removed from smoke_web.sh and every validate_*.sh; CAM= now overrides the camera they exercise. - setup_mediamtx.sh used GNU-only 'grep -oP' and always fetched linux_amd64; it now detects OS/arch and parses the tag portably (MEDIAMTX_TAG= pins one). - Publish authorization moved to the kernel (authMethod: http -> /internal/mediamtx-auth), so an ffmpeg publisher started before heldar-core gets a 401 and exits. validate.sh, smoke_web.sh and e2e_stack.sh all started publishers first, so they were validating cameras that never streamed. They now publish after the API is healthy; validate.sh aborts if its camera dies. - e2e_stack.sh additionally ran the core on :8011 while mediamtx.yml pins the auth callback to :8000, so MediaMTX denied every publish and read: it now starts MediaMTX from a port-adjusted copy. Plus two macOS portability fixes (bash 3.2 has no negative array subscripts; fuser has no -k). Docs: LICENSING, DESIGN-PRINCIPLES #8, the open-core and module-system pages (with es/zh-Hans), REMOTE-ACCESS, PRODUCTION, README and the heldar-server composition-root comments still described the retired generated-tree model -- a public repo generated from a private monorepo, with main.rs substituted per build. That directly contradicted CONTRIBUTING and would tell a contributor their PR gets regenerated away. They now describe the real model, and ARCHITECTURE.md no longer points at a docs/adr/ path absent from this repo. Verified on a fresh clone: fmt, clippy -D warnings, 288 tests, read-seam lint, open build and the dashboard build all pass; validate.sh records 7 segments at 15fps and cuts a clip; the e2e stack brings both cameras to 'recording'. Signed-off-by: wms2537 <51080539+wms2537@users.noreply.github.com>
Shipped code and docs cited ADR 0001/0003/0004 as the design records of the open platform, but docs/adr/ did not exist here -- contributors were pointed at documents they could not read. Publishes the two that describe THIS codebase: - 0003: remote viewing over WebRTC; retiring the mobile app + kernel-managed WireGuard. Trimmed before publishing: the phase-by-phase operational log (infrastructure secret names, deployment TODOs) and the list of deliberately deferred security hardenings on the internet-facing relay -- an attacker roadmap that does not belong in public. The decision, rationale, alternatives, consequences and risks are intact. - 0004: edge nodes stay on SQLite; scale by adding nodes rather than porting to Postgres. Cross-references rewritten: it cited 'issue #36' (a dependabot PR in this repo), CLAUDE.md (a private file), and ADR 0001 (unpublished). Decisions wholly about the commercial tier stay private; docs/adr/README.md says so rather than leaving the gap unexplained. Also: services/embeddings.rs credited the no-vector-DB/no-ANN choice to ADR 0004, which is the SQLite-vs-Postgres brief -- no ADR records that decision, so the rationale now stands on its own. CONTRIBUTING notes that 'issue #NN' in comments predating 2026-07 refers to the pre-flip tracker, not this repo (65 such references, where e.g. #38 is now a setup-node bump and #46 is a live good-first-issue). Signed-off-by: wms2537 <51080539+wms2537@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A production-readiness pass run against a fresh clone of this repo — i.e. what an arriving contributor actually gets, not what works on a machine that already has everything.
The engineering gates were already green. What was broken was everything around the code: the entry path, the harnesses, and the story the docs tell.
Blockers a first-time visitor would hit
The from-source quickstart started nothing.
scripts/run_stack.sh— the last step of both the README quickstart and the CONTRIBUTING dev setup — resolved every path from a hardcoded/home/soh/cctv. With no precondition checks it failed silently, printingstack up: mediamtx=… core=… vite=…and then sleeping for 30 minutes. Reproduced verbatim:Now repo-relative, honours
HELDAR_DATA_DIR, and exits non-zero on a missing core binary / MediaMTX / dashboard deps. The same hardcoded root is gone fromsmoke_web.shand all eightvalidate_*.sh.setup_mediamtx.shonly worked on Linux/x86. It parsed the release tag with GNU-onlygrep -oPand always downloadedlinux_amd64— so it failed on a dev Mac and on the arm64 appliance target. Now detects OS/arch (linux + darwin; amd64/arm64/armv7/armv6) and parses the tag portably.The harnesses were not validating anything
Publish authorization moved to the kernel (
authMethod: http→/internal/mediamtx-auth), which means an ffmpeg publisher started before heldar-core gets a401 Unauthorizedand exits immediately.validate.sh,smoke_web.shand the Playwrighte2e_stack.shall started publishers first — so every one of them was exercising cameras that never streamed.Before / after on
validate.sh, same clone:offline, 5 reconnectsrecordingnulle2e_stack.shhad a second, independent bug: it runs the core on:8011for isolation, butmediamtx.ymlpins the auth callback to:8000, so MediaMTX asked a dead port and denied every publish and read. It now starts MediaMTX from a port-adjusted copy of the config. Both e2e cameras reachrecording, 8 segments, 10 fps.Two macOS portability fixes there as well:
wait "${PIDS[-1]}"needs bash ≥ 4.3 and aborted underset -uon macOS's bash 3.2, andfuser -khas no-kon macOS (falls back tolsof).Docs still described the retired generated-tree model
LICENSING.md,DESIGN-PRINCIPLES.md#8, the open-core and module-system pages (pluses/zh-Hans),REMOTE-ACCESS.md,PRODUCTION.md, and theheldar-servercomposition-root comments all said the public repo is generated from a private monorepo and thatmain.rsis substituted per build.That directly contradicts CONTRIBUTING (which correctly says there is no mirror and no import step) and would tell an arriving contributor their PR gets regenerated away. Rewritten to the real model: this repo is the source of truth, and a private product composes its own binary against
heldar_server::run(impl Verticals).ARCHITECTURE.md§21 also no longer points at adocs/adr/path that does not exist here.Verified on the fresh clone
cargo fmt --check·clippy --workspace --all-targets -D warnings· 288 tests · read-seam lint · open build (--no-default-features, no wasmi linked) ·npm run typecheck && npm run build· Docusaurus build — all pass.setup_mediamtx.shverified on darwin/arm64, where the old one failed twice.Also checked and clean, no change needed: no secrets in the full 234-commit history; ghcr images anonymously pullable (
0.3.0/0.3.1/latest); the advertisedinstall.shis live and byte-identical towebsite/static/install.sh, and the raw compose URLs it fetches return 200; crate metadata complete and consistent at 0.3.1; relative markdown links resolve.Not in this PR — needs a decision
SECURITY.mdpoints at a disabled feature. It directs reporters to GitHub Private Vulnerability Reporting, but that is{"enabled": false}on this repo — so the documented channel dead-ends and a researcher's only remaining option is the public issue the policy forbids.main— CI is advisory and force-push is possible.docs/adr/is absent while shipped code and docs cite ADR 0001/0003/0004.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.