Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2022,8 +2022,8 @@ A deployment is normally behind **CGNAT** (shared public IPv4, no inbound port-f
useless), so the only thing that reaches it is the node **dialing out**. Remote access is therefore
**browser-based WebRTC**: the box dials OUT to a **signaling + TURN service** in the control plane,
NAT traversal is negotiated over ICE, and a browser viewer plays live video over **WHEP** (served by
MediaMTX). Full rationale and the deploy recipes live in `docs/REMOTE-ACCESS.md` and
`docs/adr/0003-webrtc-remote-access.md`; this section records the architecture.
MediaMTX). The deploy recipes live in `docs/REMOTE-ACCESS.md`; this section is the architecture of
record, and the decision that produced it is `docs/adr/0003-webrtc-remote-access.md`.

**Two layers, kept separate.** *Reachability* is WebRTC: the box and the browser exchange SDP/ICE
through the control-plane **signaling** endpoint, then establish a direct peer connection whenever
Expand Down
50 changes: 49 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,55 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

_Nothing yet._
### Fixed

- **The from-source quickstart works in any clone.** `scripts/run_stack.sh` resolved its paths from a
hardcoded `/home/soh/cctv`, so for everyone else it started nothing — and, having no precondition
checks, still printed `stack up: …` and slept for 30 minutes. It now resolves paths relative to the
script, honours `HELDAR_DATA_DIR`, and fails loudly on a missing binary, MediaMTX, or dashboard
deps. The same hardcoded root is gone from `smoke_web.sh` and every `validate_*.sh`, whose reports
now land in the repo's `data/`; the camera they exercise is overridable with `CAM=`.

- **`scripts/setup_mediamtx.sh` runs outside Linux/x86.** It parsed the release tag with `grep -oP`
(GNU-only — it failed outright on macOS/BSD) and always downloaded `linux_amd64`. It now detects
OS/arch (linux + darwin; amd64/arm64/armv7/armv6) and parses the tag portably. `MEDIAMTX_TAG=`
pins a release.

- **Synthetic-camera harnesses publish after the core, not before.** Since publish authorization moved
to the kernel (`authMethod: http` → `/internal/mediamtx-auth`), starting an ffmpeg publisher before
heldar-core is up gets a 401 and the publisher exits immediately — so `validate.sh`, `smoke_web.sh`
and the Playwright `e2e_stack.sh` were all exercising cameras that never streamed. The publishers now
start after the API is healthy, and `validate.sh` aborts if its camera dies.

- **The Playwright e2e stack actually records.** `e2e_stack.sh` runs the core on `:8011`, but
`mediamtx.yml` pins the kernel auth callback to `:8000`, so MediaMTX asked a dead port and denied
every publish *and* read. It now starts MediaMTX from a port-adjusted copy of the config. Two further
portability fixes: `wait "${PIDS[-1]}"` needs bash ≥ 4.3 and aborted under `set -u` on macOS's bash
3.2 (it now waits on the core PID), and the `fuser -k` port cleanup falls back to `lsof` where
`fuser` has no `-k`.

### Documentation

- Docs no longer describe the retired generated-tree model. `LICENSING.md`, `DESIGN-PRINCIPLES.md` #8,
the open-core and module-system pages (plus their `es`/`zh-Hans` translations), `REMOTE-ACCESS.md`,
`PRODUCTION.md` and the `heldar-server` composition-root comments said the public repo was *generated
from a private monorepo* and that `main.rs` was substituted per build — which contradicted
CONTRIBUTING and would have told a contributor their PR gets regenerated away. They now describe the
real model: this repo is the source of truth, and a private product composes its own binary against
`heldar_server::run(impl Verticals)`.
- **Architecture decision records are published** under [`docs/adr/`](docs/adr): ADR 0003 (remote
viewing over WebRTC; retiring the mobile app and kernel-managed WireGuard) and ADR 0004 (edge nodes
stay on SQLite rather than porting to Postgres). Shipped code and docs cited these as the design
records while the directory did not exist here, so `ARCHITECTURE.md` §21 and `REMOTE-ACCESS.md` now
link to a document a reader can actually open. Decisions that are wholly about the commercial tier
stay unpublished, and the index says so.
- `services/embeddings.rs` credited the no-vector-DB/no-ANN choice to ADR 0004, which is the
SQLite-versus-Postgres brief; no ADR records that decision, so the rationale now stands on its own.
- CONTRIBUTING notes that `issue #NN` references predating 2026-07 point at the tracker used before
development moved into the open, and do not match this repo's issue numbers.
- README/CONTRIBUTING dev setup now includes the dashboard `npm ci` step that `run_stack.sh` requires,
and the README no longer describes the production overlay as switching to a private image (it is an
open hardening overlay).

## [0.3.1] - 2026-07-31

Expand Down
10 changes: 9 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,14 @@ Prerequisites: Rust (via `rustup`), FFmpeg + ffprobe on `PATH`, Node.js (fronten
```bash
rustup update
cargo build --workspace
cd apps/web && npm ci && cd - # dashboard deps
scripts/setup_mediamtx.sh # fetch the MediaMTX live-view gateway
scripts/run_stack.sh # MediaMTX + core (:8000) + web (Vite)
```

The per-stage `scripts/validate_*.sh` scripts exercise each capability end-to-end against a running
stack and write reports to `data/`.
stack and write reports to `data/` (override with `HELDAR_DATA_DIR`). Those that need a registered
camera default to `cam_192_168_0_2`; point them at yours with `CAM=<camera-id>`.

## Quality bar (CI will check these)

Expand All @@ -54,6 +56,12 @@ cargo build -p heldar-server --features smtp --locked
cd apps/web && npm ci && npm run build
```

- **Design decisions are recorded.** [`docs/adr/`](./docs/adr) holds the decision records — why the
system is the way it is, as opposed to how it works today (that's [ARCHITECTURE.md](./ARCHITECTURE.md)).
A decision that changes a seam or reverses one of these deserves a new ADR alongside the code.
- **A note on `issue #NN` in comments.** References predating 2026-07 point at the tracker this project
used before it moved development into the open; they do **not** correspond to issue numbers in this
repo. They are kept as provenance for the original design discussion — don't follow them here.
- **Architecture seams matter.** Apps plug into the kernel only through public seams (the
`DetectionConsumer` trait, `Router<AppState>` merging, a self-installed schema, the auth primitive).
Don't add app-specific knowledge to the kernel — see [ARCHITECTURE.md](./ARCHITECTURE.md).
Expand Down
6 changes: 4 additions & 2 deletions LICENSING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,7 @@ layer their specifics on top. They plug in only through the kernel's public seam
primitive. A deployment is **composed** from the open kernel + open apps + whichever proprietary app
crates that client needs (single-tenant per deployment).

See `ARCHITECTURE.md` for the seams. The public `heldar` repo is generated from the private monorepo
(this file is the licensing statement of record), and the open crates are published to crates.io.
See `ARCHITECTURE.md` for the seams. This repository **is** the source of truth for everything
Apache-2.0 above (this file is the licensing statement of record); the open crates are published to
crates.io from here. Proprietary products live in their own private repositories and consume the
published crates through the documented seams — they are never merged into this tree.
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ curl -fsSL https://heldar.swmengappdev.workers.dev/install.sh | sh

Pulls the prebuilt **OPEN** images (kernel + generic apps) and starts MediaMTX + core + web — the
dashboard is then at `http://localhost:8080`. Add the reference AI worker with `--profile ai`; update
with `docker compose pull`. Production (private full image, auth, secrets, TLS) uses the overlay
`docker compose -f deploy/compose.yml -f deploy/compose.prod.yml up -d` — see
with `docker compose pull`. For production (auth on, secure cookies, strict boot guardrails) layer the
hardening overlay: `docker compose -f deploy/compose.yml -f deploy/compose.prod.yml up -d` — see
[`docs/PRODUCTION.md`](docs/PRODUCTION.md). For a flashed DVR/appliance, use the native-systemd image
instead (`make appliance-image`, [`infra/systemd/`](infra/systemd/)).

Expand All @@ -51,6 +51,7 @@ dashboard; Python 3 for the AI worker.
rustup update # the project tracks latest stable
cargo build --workspace
cp .env.example .env # defaults work out of the box; never commit .env
(cd apps/web && npm ci) # dashboard dependencies
scripts/setup_mediamtx.sh # fetch the MediaMTX live-view gateway
scripts/run_stack.sh # MediaMTX + core (http://localhost:8000) + web (Vite on :5173)
```
Expand Down
2 changes: 1 addition & 1 deletion crates/heldar-kernel/src/services/embeddings.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
//! worker's `embedding` task, the pull-only query-embedding job queue, and brute-force cosine
//! top-k over the stored vectors.
//!
//! Vectors live as little-endian f32 BLOBs in SQLite (no vector DB / ANN index — ADR 0004): at
//! Vectors live as little-endian f32 BLOBs in SQLite (deliberately no vector DB / ANN index): at
//! single-box scale even a million 512-d vectors scan in tens of milliseconds, and the scan
//! streams rows so peak memory stays at one row + the k-sized heap.
//!
Expand Down
9 changes: 5 additions & 4 deletions crates/heldar-server/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
//! The composed `heldar-core` binary for THIS workspace: the library server plus the in-tree
//! vertical composition (`verticals.rs` — the real proprietary module here; a no-op stub in the
//! open repo, so this file is identical across both). An out-of-tree overlay builds its own bin
//! against `heldar_server::run` instead of this one.
//! The composed `heldar-core` binary: the library server (`heldar_server::run`) plus this
//! workspace's vertical composition (`verticals.rs`, a no-op here — the open build links no
//! proprietary code). A private product does NOT fork this file: it depends on this crate (by git
//! tag — the composition crate is `publish = false`) and calls `heldar_server::run` with its own
//! `Verticals` implementation.

use axum::Router;
use heldar_kernel::modules::ModuleManifest;
Expand Down
9 changes: 5 additions & 4 deletions crates/heldar-server/src/verticals.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
//! Proprietary vertical composition seam — a NO-OP stub in the open repo.
//! Vertical composition for this workspace — deliberately a NO-OP.
//!
//! `main.rs` calls these functions unconditionally. In the open build they do nothing and reference
//! no proprietary crate. The private workspace replaces this file with the real composition module
//! (the proprietary verticals) — `main.rs` is identical across both repos.
//! `main.rs` adapts these functions into a [`heldar_server::Verticals`] implementation. Nothing here
//! references a proprietary crate, so the reference `heldar-core` binary links none. This file is
//! also the smallest worked example of the seam: a private product implements the same four hooks
//! in its own repository and passes them to `heldar_server::run`.

use axum::Router;
use heldar_kernel::modules::ModuleManifest;
Expand Down
10 changes: 6 additions & 4 deletions docs/DESIGN-PRINCIPLES.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,12 @@ explicitly and why.
one. Deploys keep a rollback binary. A cleared runtime override reverts to the env default. Prefer
changes that can be undone without a data migration.

8. **Open-core discipline.** The public repo is *generated* from the private monorepo by a scrubbing
step; proprietary code and names never reach it, and secrets are never committed (reference
credentials by `file:line` + type only). If a change could leak proprietary material to the open
tree, the generator must strip it and the leak-gate must catch it.
8. **Open-core discipline.** This repository is the source of truth for the open platform, and it is
developed in the open — the commit that lands here is the commit that ships. Proprietary code and
client names therefore must never be written into it in the first place, and secrets are never
committed (reference credentials by `file:line` + type only). Anything vertical- or
client-specific belongs in its own private repository, consuming these crates through the
published seams.

9. **Compose, don't couple.** Apps plug into the kernel through narrow, named seams — a
`DetectionConsumer`, a `Router<AppState>` merge, a self-installed schema (`schema::init`). Adding a
Expand Down
3 changes: 1 addition & 2 deletions docs/PRODUCTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ or an empty dial-out bearer (`HELDAR_CP_TOKEN`) while a rendezvous is configured

Each tagged release attaches static `heldar-core` binaries (x86_64 + aarch64). To upgrade a box in place:

REPO=Straits-AI/heldar # open build (self-hosters)
# REPO=Straits-AI/heldar-proprietary # full/licensed build
REPO=Straits-AI/heldar # licensed/full deployments substitute their own release repo
ARCH=$(uname -m) # x86_64 or aarch64
V=vX.Y.Z
curl -fsSLO "https://github.com/$REPO/releases/download/$V/heldar-core-$V-$ARCH-linux-musl"
Expand Down
12 changes: 6 additions & 6 deletions docs/REMOTE-ACCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ common case for home/small-site internet: a shared public IPv4, no inbound port-
useless). This is an **open kernel** capability: every deployment of the Apache-2.0 kernel gets
private remote viewing out of the box.

Remote access is **WebRTC-primary, browser-based** — see [`ARCHITECTURE.md`](../ARCHITECTURE.md) §21
(the WebRTC model; the full design of record is ADR 0003, `docs/adr/0003-webrtc-remote-access.md` in
the private monorepo). All phases are **shipped** (P1 live video → P2 universal reach → P3 the full dashboard; see
_Status & phasing_ below). The optional self-hoster **overlay** paths (Recipes A/B) remain available
Remote access is **WebRTC-primary, browser-based** — the design of record is
[`ARCHITECTURE.md`](../ARCHITECTURE.md) §21, which carries the full WebRTC model; the decision that
produced it is [ADR 0003](adr/0003-webrtc-remote-access.md). All phases are **shipped** (P1 live
video → P2 universal reach → P3 the full dashboard; see _Status & phasing_ below). The optional self-hoster **overlay** paths (Recipes A/B) remain available
for operators who prefer full-L3 reach over the hosted rendezvous. Hardening a deployment for the
public internet: [`docs/PRODUCTION.md`](PRODUCTION.md).

Expand All @@ -21,7 +21,7 @@ public internet: [`docs/PRODUCTION.md`](PRODUCTION.md).
**signaling + TURN hosted in `heldar-control-plane`**; live video rides **MediaMTX / WHEP**
(`:8889`). Media is **end-to-end encrypted (DTLS-SRTP)**: the rendezvous brokers only SDP/ICE and
relayed control, never the video bytes. Design: [`ARCHITECTURE.md`](../ARCHITECTURE.md) §21
(ADR 0003).
([ADR 0003](adr/0003-webrtc-remote-access.md)).
- **Optional (works today): a WireGuard overlay** running as an external daemon on the host, for
self-hosters who want full L3 reachability rather than just the browser view.
- **Your own / dev use → Tailscale** (Personal, free): zero servers, near-zero ops, $0.
Expand Down Expand Up @@ -85,7 +85,7 @@ Headscale) removes even that third-party metadata, at the cost of a small VPS to
## Status & phasing

WebRTC remote access **shipped in three phases, all landed** (implementation:
[`ARCHITECTURE.md`](../ARCHITECTURE.md) §21; full phased plan: ADR 0003):
[`ARCHITECTURE.md`](../ARCHITECTURE.md) §21; the decision: [ADR 0003](adr/0003-webrtc-remote-access.md)):

- **P1 — LAN / WHEP ✅:** sub-second live video in the browser over MediaMTX WHEP (`:8889`) on the LAN.
- **P2 — universal reach ✅:** the box dials out to **signaling + TURN**, so the same browser view works
Expand Down
Loading
Loading