From 122d48503a25547a8d4b836d1d60af36f1c4dee9 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 09:46:56 -0400 Subject: [PATCH 01/13] docs: acquisition refactor design (CONTEXT, ADRs 0001-0003, Phase 1 plan) Co-Authored-By: Claude Opus 4.8 --- CONTEXT.md | 39 + ...uple-acquisition-from-http-and-tracking.md | 66 + docs/adr/0002-hold-focus-not-autofocus.md | 52 + docs/adr/0003-recording-integrity.md | 38 + .../2026-06-21-acquisition-core-refactor.md | 1313 +++++++++++++++++ 5 files changed, 1508 insertions(+) create mode 100644 CONTEXT.md create mode 100644 docs/adr/0001-decouple-acquisition-from-http-and-tracking.md create mode 100644 docs/adr/0002-hold-focus-not-autofocus.md create mode 100644 docs/adr/0003-recording-integrity.md create mode 100644 docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 0000000..7a88d92 --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,39 @@ +# WormsPy — Context & Glossary + +A glossary of the domain language used in WormsPy. Implementation details live in code and ADRs, not here. + +## Glossary + +### Frame-synced recording +The guarantee that a frame from the **brightfield/behaviour** camera and a frame from the +**fluorescence/calcium** camera can be reliably paired for offline analysis (e.g. correlating +RIA compartmental calcium with head-bending). + +Two distinct properties are bundled under this term; keep them separate: + +- **Sensor-level sync** — the two sensors expose at the same instant. Currently **not** provided + (cameras free-run). Achievable via FLIR GPIO master–slave hardware trigger. Considered + *desirable but optional*: **tens of ms of offset between the two streams is acceptable** for the + current science. +- **Loop coherence (no drift)** — the pairing between the two streams must **not drift without + bound** over a recording. This is a **hard requirement**. The current two-independent-generator + design violates it. + +### Hold Focus (formerly "FocusLock" / "autofocus") +Keeping the neuron of interest in focus on the fluorescence/calcium feed. True autofocus is +**not achievable** on this rig: +- There is **one shared z motor and one objective**, so both cameras focus at a single plane — + you cannot independently focus brightfield and fluorescence. +- Best focus is defined by the **target neuron**, whose depth inside the worm **changes every + session**, so there is no fixed parfocal offset to calibrate once. + +Therefore focus is **set by the operator** (manually, via joystick, watching the fluorescence +feed) and software only **holds or restores** that z. An optional, opt-in **drift compensation** +mode may actively nudge z using a brightfield reference captured at focus-set time, at the cost of +a focus/intensity artifact in the recording (see [[ADR 0002]]). + +### Loop drift +The accumulating divergence between the brightfield and fluorescence acquisition loops because +each runs its own independent `while cap.isOpened()` loop with its own wall-clock timestamps and +its own (possibly different, possibly dropped-frame) cadence. **Unacceptable** — see +[[frame-synced-recording]]. diff --git a/docs/adr/0001-decouple-acquisition-from-http-and-tracking.md b/docs/adr/0001-decouple-acquisition-from-http-and-tracking.md new file mode 100644 index 0000000..efba302 --- /dev/null +++ b/docs/adr/0001-decouple-acquisition-from-http-and-tracking.md @@ -0,0 +1,66 @@ +# 1. Decouple acquisition from the HTTP stream and from tracking + +Date: 2026-06-21 + +## Status + +Accepted + +## Context + +In the original design, frame acquisition for each camera lived *inside* a Flask +MJPEG generator (`video_feed`, `video_feed_fluorescent`). Each generator ran its +own `while cap.isOpened()` loop, opened its own camera, and the left generator also +opened the Zaber serial connection and ran the worm-tracking control inline. + +Consequences of that design: + +- The two camera loops were driven by how fast each browser `` consumed the + MJPEG stream, so they free-ran independently and could drift without bound. This + violates the hard requirement that the two streams stay paired (see + [[loop-drift]] in CONTEXT.md). +- The Zaber motors and `xMotor/yMotor/zMotor` globals only existed while the left + feed was streaming, so `move_to_center`, manual mode, and autofocus crashed + (`NameError`) otherwise, and the motor connection died on browser disconnect. +- Recorded playback speed depended on a hand-typed `FPS` constant rather than real + capture timing. + +The science needs a **fixed, stable acquisition frame rate with constant exposure** +for the calcium recording above all else. Tracking stability is explicitly lower +priority and may lag. + +## Decision + +1. **Single acquisition loop, owned by the application (not the HTTP handler).** + One loop grabs both cameras, assigns each pair a shared monotonically-increasing + `frame_index` plus capture timestamp + camera frame ID, writes to the recording, + and publishes the latest frame of each camera for display. Frame pairing is + guaranteed by construction; drift is structurally impossible. + +2. **The camera is the clock.** Acquisition rate and exposure are fixed camera-side + (`AcquisitionFrameRateEnable` + fixed `AcquisitionFrameRate`, auto-exposure/gain + off). Recording uses a queued grab strategy so every sensor frame is written; + per-frame camera frame ID + device timestamp are logged so dropped frames appear + as index gaps rather than silent timing errors. `LatestImageOnly` is used only for + the display publish. This is the natural bridge to an optional external hardware + trigger later. + +3. **The Zaber connection opens once at application startup** and is held for the + process lifetime, degrading gracefully if no hardware is present. + +4. **Tracking runs in its own thread for all tracking modes** (thresholding, + fluorescent marker, DeepLabCut). It consumes the latest brightfield frame, + computes worm position, and commands the motors. Fast modes keep up frame-for-frame; + DLC may lag but never affects the acquisition/recording cadence. This supersedes an + earlier consideration of tracking inline in the acquisition loop. + +## Consequences + +- The non-negotiable (paired frames at a fixed, stable rate and exposure for the + calcium recording) is met regardless of tracking cost. +- Tracking control may lag the live frame by one or more frames, especially under + DLC. Acceptable: a worm does not teleport between frames and motor control tolerates + lag far better than the recording tolerates a variable rate. +- A "latest frame" hand-off between acquisition and both the display endpoints and the + tracker becomes a shared-state surface that must be made thread-safe. +- Motor absence at startup must be handled so the server still runs for UI/dev work. diff --git a/docs/adr/0002-hold-focus-not-autofocus.md b/docs/adr/0002-hold-focus-not-autofocus.md new file mode 100644 index 0000000..e814177 --- /dev/null +++ b/docs/adr/0002-hold-focus-not-autofocus.md @@ -0,0 +1,52 @@ +# 2. "Hold Focus" instead of automatic focus + +Date: 2026-06-21 + +## Status + +Accepted + +## Context + +The original `FocusLock`/autofocus drove the z-stage with a PID whose setpoint was the +Laplacian-variance focus metric captured at the instant the user enabled it, reading frames +from the fluorescence camera. This is broken: sharpness vs. z is an unsigned unimodal hill (a +PID cannot hill-climb it), the setpoint was an arbitrary starting value rather than the +in-focus peak, and the metric was computed on the wrong camera. + +Attempting to redesign it surfaced hard physical constraints of the rig: + +- **One shared z motor and one objective.** Both cameras image a single focal plane; you + cannot keep brightfield and fluorescence in focus at two different planes at once. +- **Best focus is defined by the target neuron**, whose depth within the worm changes every + recording session. There is no fixed parfocal offset to calibrate once; per-session + calibration was judged infeasible. +- The neuron is often only a few bright pixels in an otherwise black/low-signal frame, so a + whole-frame focus metric on the fluorescence channel is dominated by background noise. +- Any active focus loop dithers the single shared stage, and that dither couples directly into + the calcium intensity signal — a confound that is worse the thinner the depth of field. + +## Decision + +Focus is **set by the operator** (manual joystick z control while watching the fluorescence +feed). Software does not compute focus. Two behaviours are provided: + +- **(A) Hold / Restore focus (default).** A "Set Focus" action stores the current z; the stage + holds it and can lock out accidental z drift during recording; a "Return to Focus" action + snaps back to the stored z. No sensing, no dither, no artifact. +- **(B) Drift compensation (opt-in, experimental).** At focus-set time, capture the brightfield + focus state as a reference (implicitly capturing the session's offset, no separate + calibration). During recording a small brightfield hill-climb nudges z to keep the worm body + at that reference, tracking z-drift on the assumption the neuron-to-body geometry holds. Logs + z per frame. Clearly labelled as introducing a focus/intensity artifact. + +The UI is renamed from "FocusLock"/"autofocus" to "Hold Focus" so the name does not promise +behaviour the hardware cannot deliver. + +## Consequences + +- The default path produces the cleanest possible calcium trace (no stage motion during + recording). +- Worm z-drift during a recording is handled by the operator (default) or by opt-in (B) at a + known cost. +- Removes the PID controller and the fluorescence-channel focus metric entirely. diff --git a/docs/adr/0003-recording-integrity.md b/docs/adr/0003-recording-integrity.md new file mode 100644 index 0000000..3f95733 --- /dev/null +++ b/docs/adr/0003-recording-integrity.md @@ -0,0 +1,38 @@ +# 3. Recording integrity: CSV spine and intra-frame brightfield codec + +Date: 2026-06-21 + +## Status + +Accepted + +## Context + +The original recording wrote the brightfield camera as an XVID-compressed AVI and the +fluorescence camera as uncompressed 16-bit TIFF. A separate CSV logged only +`timestamp, X_position, Y_position` from the left/brightfield loop, with no link between a CSV +row and a specific video frame, and playback speed depended on a hand-typed `FPS` constant. +The README and code also disagreed on the brightfield codec (README said MJPG, code used XVID). + +The paper's analysis aligns calcium activity with stage motion and behaviour, so frame-accurate +addressability and frame-accurate behaviour scoring matter. + +## Decision + +- **The CSV is the per-frame spine of a session.** One row per acquisition tick, keyed by a + shared `frame_index`, logging at minimum: `frame_index, capture_timestamp, left_frame_id, + right_frame_id, x_pos, y_pos, z_pos, tracking_mode, is_tracking`. Every video frame is + addressable by `frame_index`; dropped frames appear as index gaps. (Reserve room for stimulus + state once closed-loop optogenetics lands.) +- **Brightfield uses an intra-frame codec** (MJPG, or FFV1 for lossless) rather than the + interframe XVID, so each frame stands alone and behaviour scoring never reads interpolated + content. Lossless is offered as an option alongside the existing TIFF path. README and code are + reconciled. +- Fluorescence remains uncompressed 16-bit TIFF to preserve dynamic range. + +## Consequences + +- Sessions become self-describing and frame-accurate; the `FPS` constant is no longer trusted for + timing (see [[ADR 0001]]). +- Brightfield files are larger than XVID but frame-accurate. +- Downstream analysis joins video to telemetry on `frame_index` rather than wall-clock guesswork. diff --git a/docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md b/docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md new file mode 100644 index 0000000..1335da0 --- /dev/null +++ b/docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md @@ -0,0 +1,1313 @@ +# Acquisition Core Refactor (ADR 0001) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Move camera acquisition out of the Flask MJPEG handlers into a single application-owned acquisition loop that grabs both cameras in lockstep, assigns a shared monotonic `frame_index`, publishes the latest frame for display, and feeds a tracking thread — eliminating inter-camera loop drift and the "motors die when the browser disconnects" failure class. + +**Architecture:** A small `core` package introduces three seams that can be faked in tests — `Camera`, `Stage`, and a thread-safe `FrameHub` — plus an `AcquisitionEngine` that orchestrates them and a `Tracker` thread that consumes the hub. Pure logic and orchestration are unit-tested against fakes on the dev machine; the hardware adapters (`ZaberStage`, `SpinnakerCamera`) are thin and verified manually on the rig. `app.py` becomes thin wiring: it constructs the engine at startup and exposes display endpoints that read from the hub. + +**Tech Stack:** Python 3.8 (rig) / 3.12 (dev), Flask, NumPy, OpenCV, pytest. Hardware libs (`EasyPySpin`, `pypylon`, `zaber_motion`) are imported only inside adapter modules so tests never load them. + +**Scope:** This is Phase 1 of a 4-phase roadmap (see `docs/adr/0001-0003`). It deliberately does NOT implement the new recording format (Phase 2 / ADR 0003), the Hold Focus redesign (Phase 3 / ADR 0002), or the `/status` endpoint and UI changes (Phase 4 / Q6). Recording and focus are left calling their existing code paths, adapted minimally to read frames from the hub, so the app keeps working after each task. + +--- + +## File Structure + +All paths are relative to `WormSpy/backend/code/`. + +- Create: `core/__init__.py` — package marker. +- Create: `core/frames.py` — `Frame` and `AcquiredPair` dataclasses; the `FrameHub` thread-safe latest-frame store. +- Create: `core/cameras.py` — `Camera` ABC, `FakeCamera` (test double). Hardware adapter `SpinnakerCamera` added in Task 8. +- Create: `core/stage.py` — `Stage` ABC, `clamp_native()` pure helper, `NullStage`, `FakeStage`. Hardware adapter `ZaberStage` + `connect_stage()` factory added in Task 7. +- Create: `core/acquisition.py` — `AcquisitionEngine`. +- Create: `core/tracking.py` — pure tracking helpers moved from `app.py` (`thresh_light_background`, `thresh_fluorescent_marker`, `find_worm_cms`, `simple_to_center`) plus the `Tracker` thread. +- Create: `tests/conftest.py`, `tests/test_frames.py`, `tests/test_cameras.py`, `tests/test_stage.py`, `tests/test_acquisition.py`, `tests/test_tracking.py`. +- Create: `requirements-dev.txt`, `pytest.ini`. +- Modify: `app.py` — remove acquisition/motor setup from `video_feed`/`video_feed_fluorescent`; construct engine + stage at startup; serve display from the hub. + +--- + +### Task 1: Test harness and package skeleton + +**Files:** +- Create: `WormSpy/backend/code/core/__init__.py` +- Create: `WormSpy/backend/code/tests/conftest.py` +- Create: `WormSpy/backend/code/pytest.ini` +- Create: `WormSpy/backend/code/requirements-dev.txt` + +- [ ] **Step 1: Create the package marker** + +Create `core/__init__.py`: + +```python +"""WormsPy acquisition core: hardware-independent acquisition, tracking, and stage control.""" +``` + +- [ ] **Step 2: Create pytest config** + +Create `pytest.ini`: + +```ini +[pytest] +testpaths = tests +python_files = test_*.py +addopts = -v +``` + +- [ ] **Step 3: Create conftest to make `core` importable from the code dir** + +Create `tests/conftest.py`: + +```python +import os +import sys + +# Ensure `import core...` resolves when pytest is run from WormSpy/backend/code +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +``` + +- [ ] **Step 4: Pin dev-only deps** + +Create `requirements-dev.txt`: + +```text +pytest>=7.0 +numpy>=1.20 +``` + +- [ ] **Step 5: Verify pytest collects an empty suite** + +Run: `cd WormSpy/backend/code && python -m pytest` +Expected: exit code 5 ("no tests ran") — confirms config is valid and collection works. + +- [ ] **Step 6: Commit** + +```bash +git add WormSpy/backend/code/core/__init__.py WormSpy/backend/code/pytest.ini WormSpy/backend/code/tests/conftest.py WormSpy/backend/code/requirements-dev.txt +git commit -m "test: add pytest harness and core package skeleton" +``` + +--- + +### Task 2: Frame types and the thread-safe FrameHub + +**Files:** +- Create: `WormSpy/backend/code/core/frames.py` +- Test: `WormSpy/backend/code/tests/test_frames.py` + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_frames.py`: + +```python +import threading +import numpy as np +from core.frames import Frame, AcquiredPair, FrameHub + + +def _frame(val, fid=0, ts=0.0): + img = np.full((4, 4), val, dtype=np.uint8) + return Frame(image=img, device_frame_id=fid, capture_ts=ts) + + +def test_hub_returns_none_before_first_publish(): + hub = FrameHub() + assert hub.latest("left") is None + + +def test_hub_returns_most_recent_frame(): + hub = FrameHub() + hub.publish("left", _frame(1, fid=1)) + hub.publish("left", _frame(2, fid=2)) + latest = hub.latest("left") + assert latest.device_frame_id == 2 + assert int(latest.image[0, 0]) == 2 + + +def test_hub_keeps_streams_independent(): + hub = FrameHub() + hub.publish("left", _frame(1)) + hub.publish("right", _frame(9)) + assert int(hub.latest("left").image[0, 0]) == 1 + assert int(hub.latest("right").image[0, 0]) == 9 + + +def test_hub_is_thread_safe_under_concurrent_writes(): + hub = FrameHub() + def writer(start): + for i in range(start, start + 500): + hub.publish("left", _frame(i % 256, fid=i)) + threads = [threading.Thread(target=writer, args=(s,)) for s in (0, 1000)] + for t in threads: + t.start() + for t in threads: + t.join() + # No exception and a valid frame is present + assert hub.latest("left") is not None + + +def test_acquired_pair_fields(): + pair = AcquiredPair(frame_index=7, capture_ts=1.5, left=_frame(1), right=_frame(2)) + assert pair.frame_index == 7 + assert pair.capture_ts == 1.5 + assert pair.left.device_frame_id == 0 + assert pair.right is not None +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_frames.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.frames'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/frames.py`: + +```python +import threading +from dataclasses import dataclass +from typing import Dict, Optional + +import numpy as np + + +@dataclass +class Frame: + """A single grabbed image plus the metadata needed to align it offline.""" + image: np.ndarray + device_frame_id: int # camera-provided frame id; -1 if the camera cannot supply one + capture_ts: float # time.monotonic() captured as close to the grab as possible + + +@dataclass +class AcquiredPair: + """One acquisition tick: a brightfield and a fluorescence frame sharing an index.""" + frame_index: int # monotonic counter assigned by the engine, contiguous, never reused + capture_ts: float # engine timestamp for the tick (time.monotonic()) + left: Optional[Frame] # brightfield/behaviour; None if that grab failed this tick + right: Optional[Frame] # fluorescence/calcium; None if that grab failed this tick + + +class FrameHub: + """Thread-safe store of the latest frame per named stream, for display and tracking.""" + + def __init__(self) -> None: + self._lock = threading.Lock() + self._latest: Dict[str, Frame] = {} + + def publish(self, name: str, frame: Frame) -> None: + with self._lock: + self._latest[name] = frame + + def latest(self, name: str) -> Optional[Frame]: + with self._lock: + return self._latest.get(name) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_frames.py` +Expected: PASS (5 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/frames.py WormSpy/backend/code/tests/test_frames.py +git commit -m "feat: add Frame/AcquiredPair types and thread-safe FrameHub" +``` + +--- + +### Task 3: Camera interface and FakeCamera + +**Files:** +- Create: `WormSpy/backend/code/core/cameras.py` +- Test: `WormSpy/backend/code/tests/test_cameras.py` + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_cameras.py`: + +```python +import numpy as np +from core.cameras import Camera, FakeCamera + + +def test_fake_camera_yields_scripted_frames_in_order(): + cam = FakeCamera(images=[np.full((2, 2), 1, np.uint8), + np.full((2, 2), 2, np.uint8)]) + f0 = cam.read() + f1 = cam.read() + assert int(f0.image[0, 0]) == 1 + assert int(f1.image[0, 0]) == 2 + + +def test_fake_camera_assigns_incrementing_device_frame_ids(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)] * 3) + ids = [cam.read().device_frame_id for _ in range(3)] + assert ids == [0, 1, 2] + + +def test_fake_camera_can_simulate_dropped_frames(): + # device_frame_ids 0,1,3 -> a gap (2 dropped) that downstream code can detect + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)] * 3, frame_ids=[0, 1, 3]) + ids = [cam.read().device_frame_id for _ in range(3)] + assert ids == [0, 1, 3] + + +def test_fake_camera_returns_none_when_exhausted(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)]) + assert cam.read() is not None + assert cam.read() is None + + +def test_fake_camera_read_failure_returns_none(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)], fail_on={0}) + assert cam.read() is None + + +def test_fake_camera_is_open_until_released(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)]) + assert cam.is_open() is True + cam.release() + assert cam.is_open() is False + + +def test_fake_camera_satisfies_camera_interface(): + assert issubclass(FakeCamera, Camera) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_cameras.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.cameras'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/cameras.py`: + +```python +import time +from abc import ABC, abstractmethod +from typing import Iterable, List, Optional, Set + +import numpy as np + +from core.frames import Frame + + +class Camera(ABC): + """Acquisition interface. Adapters wrap real SDKs; FakeCamera is the test double.""" + + @abstractmethod + def read(self) -> Optional[Frame]: + """Grab the next frame, or return None if the grab failed / stream ended.""" + + @abstractmethod + def is_open(self) -> bool: + ... + + @abstractmethod + def release(self) -> None: + ... + + +class FakeCamera(Camera): + """Deterministic camera for tests. Emits a scripted list of images.""" + + def __init__(self, images: Iterable[np.ndarray], + frame_ids: Optional[List[int]] = None, + fail_on: Optional[Set[int]] = None) -> None: + self._images: List[np.ndarray] = list(images) + self._frame_ids = frame_ids if frame_ids is not None else list(range(len(self._images))) + self._fail_on = fail_on or set() + self._pos = 0 + self._open = True + + def read(self) -> Optional[Frame]: + if not self._open or self._pos >= len(self._images): + return None + idx = self._pos + self._pos += 1 + if idx in self._fail_on: + return None + return Frame(image=self._images[idx], + device_frame_id=self._frame_ids[idx], + capture_ts=time.monotonic()) + + def is_open(self) -> bool: + return self._open + + def release(self) -> None: + self._open = False +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_cameras.py` +Expected: PASS (7 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/cameras.py WormSpy/backend/code/tests/test_cameras.py +git commit -m "feat: add Camera interface and FakeCamera test double" +``` + +--- + +### Task 4: Stage interface, clamp helper, Null/Fake stages + +**Files:** +- Create: `WormSpy/backend/code/core/stage.py` +- Test: `WormSpy/backend/code/tests/test_stage.py` + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_stage.py`: + +```python +import pytest +from core.stage import Stage, NullStage, FakeStage, clamp_native + + +def test_clamp_within_bounds_is_unchanged(): + value, hit = clamp_native(50.0, 0.0, 100.0) + assert value == 50.0 + assert hit is False + + +def test_clamp_above_max_pins_to_max_and_flags(): + value, hit = clamp_native(150.0, 0.0, 100.0) + assert value == 100.0 + assert hit is True + + +def test_clamp_below_min_pins_to_min_and_flags(): + value, hit = clamp_native(-5.0, 0.0, 100.0) + assert value == 0.0 + assert hit is True + + +def test_null_stage_is_not_connected(): + stage = NullStage() + assert stage.is_connected() is False + + +def test_null_stage_moves_are_noops(): + stage = NullStage() + # Must not raise even though there is no hardware + stage.move_relative("x", 100.0) + assert stage.get_position("x") == 0.0 + + +def test_fake_stage_tracks_relative_moves_and_clamps(): + stage = FakeStage(limits={"x": (0.0, 100.0)}, positions={"x": 90.0}) + hit = stage.move_relative("x", 50.0) # 90 + 50 = 140 -> clamp to 100 + assert stage.get_position("x") == 100.0 + assert hit is True + + +def test_fake_stage_relative_move_within_bounds_does_not_flag(): + stage = FakeStage(limits={"x": (0.0, 100.0)}, positions={"x": 10.0}) + hit = stage.move_relative("x", 5.0) + assert stage.get_position("x") == 15.0 + assert hit is False + + +def test_fake_stage_satisfies_stage_interface(): + assert issubclass(FakeStage, Stage) + assert issubclass(NullStage, Stage) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_stage.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.stage'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/stage.py`: + +```python +from abc import ABC, abstractmethod +from typing import Dict, Tuple + + +def clamp_native(value: float, minimum: float, maximum: float) -> Tuple[float, bool]: + """Clamp value into [minimum, maximum]. Returns (clamped_value, limit_was_hit).""" + if value > maximum: + return maximum, True + if value < minimum: + return minimum, True + return value, False + + +class Stage(ABC): + """XYZ stage interface in native (microstep) units. Adapters wrap real hardware.""" + + @abstractmethod + def is_connected(self) -> bool: + ... + + @abstractmethod + def get_position(self, axis: str) -> float: + ... + + @abstractmethod + def limits(self, axis: str) -> Tuple[float, float]: + ... + + @abstractmethod + def move_relative(self, axis: str, native: float) -> bool: + """Move axis by `native` microsteps, clamped to limits. Returns True if a limit was hit.""" + + @abstractmethod + def move_absolute(self, axis: str, native: float) -> bool: + """Move axis to absolute `native` microsteps, clamped to limits. Returns True if hit.""" + + +class NullStage(Stage): + """Used when no hardware is present so the app still runs. All moves are no-ops.""" + + def is_connected(self) -> bool: + return False + + def get_position(self, axis: str) -> float: + return 0.0 + + def limits(self, axis: str) -> Tuple[float, float]: + return (0.0, 0.0) + + def move_relative(self, axis: str, native: float) -> bool: + return False + + def move_absolute(self, axis: str, native: float) -> bool: + return False + + +class FakeStage(Stage): + """In-memory stage for tests. Honors limits via clamp_native.""" + + def __init__(self, limits: Dict[str, Tuple[float, float]], + positions: Dict[str, float]) -> None: + self._limits = dict(limits) + self._pos = dict(positions) + + def is_connected(self) -> bool: + return True + + def get_position(self, axis: str) -> float: + return self._pos[axis] + + def limits(self, axis: str) -> Tuple[float, float]: + return self._limits[axis] + + def move_relative(self, axis: str, native: float) -> bool: + lo, hi = self._limits[axis] + target, hit = clamp_native(self._pos[axis] + native, lo, hi) + self._pos[axis] = target + return hit + + def move_absolute(self, axis: str, native: float) -> bool: + lo, hi = self._limits[axis] + target, hit = clamp_native(native, lo, hi) + self._pos[axis] = target + return hit +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_stage.py` +Expected: PASS (8 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/stage.py WormSpy/backend/code/tests/test_stage.py +git commit -m "feat: add Stage interface, clamp helper, Null/Fake stages" +``` + +--- + +### Task 5: AcquisitionEngine (the core) + +**Files:** +- Create: `WormSpy/backend/code/core/acquisition.py` +- Test: `WormSpy/backend/code/tests/test_acquisition.py` + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_acquisition.py`: + +```python +import numpy as np +from core.cameras import FakeCamera +from core.frames import FrameHub +from core.acquisition import AcquisitionEngine + + +def _imgs(n, base=0): + return [np.full((2, 2), (base + i) % 256, np.uint8) for i in range(n)] + + +def test_run_once_assigns_contiguous_monotonic_indices(): + left = FakeCamera(_imgs(3, base=0)) + right = FakeCamera(_imgs(3, base=100)) + engine = AcquisitionEngine(left, right, FrameHub()) + indices = [] + for _ in range(3): + pair = engine.run_once() + indices.append(pair.frame_index) + assert indices == [0, 1, 2] + + +def test_run_once_pairs_left_and_right_per_tick(): + left = FakeCamera(_imgs(2, base=0)) + right = FakeCamera(_imgs(2, base=100)) + engine = AcquisitionEngine(left, right, FrameHub()) + pair = engine.run_once() + assert int(pair.left.image[0, 0]) == 0 + assert int(pair.right.image[0, 0]) == 100 + + +def test_run_once_publishes_latest_to_hub(): + hub = FrameHub() + left = FakeCamera(_imgs(1, base=5)) + right = FakeCamera(_imgs(1, base=50)) + engine = AcquisitionEngine(left, right, hub) + engine.run_once() + assert int(hub.latest("left").image[0, 0]) == 5 + assert int(hub.latest("right").image[0, 0]) == 50 + + +def test_run_once_feeds_the_sink(): + received = [] + left = FakeCamera(_imgs(2)) + right = FakeCamera(_imgs(2, base=100)) + engine = AcquisitionEngine(left, right, FrameHub(), sink=received.append) + engine.run_once() + engine.run_once() + assert [p.frame_index for p in received] == [0, 1] + + +def test_one_camera_failing_still_increments_index_and_keeps_other(): + # Left fails on tick 0; right is fine. The pair has left=None but a valid index + right. + left = FakeCamera(_imgs(2), fail_on={0}) + right = FakeCamera(_imgs(2, base=100)) + engine = AcquisitionEngine(left, right, FrameHub()) + pair0 = engine.run_once() + assert pair0.frame_index == 0 + assert pair0.left is None + assert pair0.right is not None + pair1 = engine.run_once() + assert pair1.frame_index == 1 + assert pair1.left is not None + + +def test_device_frame_ids_are_preserved_for_drop_detection(): + left = FakeCamera(_imgs(3), frame_ids=[0, 1, 3]) # camera dropped id 2 + right = FakeCamera(_imgs(3, base=100), frame_ids=[0, 1, 2]) + engine = AcquisitionEngine(left, right, FrameHub()) + left_ids = [engine.run_once().left.device_frame_id for _ in range(3)] + assert left_ids == [0, 1, 3] + + +def test_run_loop_stops_and_drains_until_stream_ends(): + left = FakeCamera(_imgs(4)) + right = FakeCamera(_imgs(4, base=100)) + sink = [] + engine = AcquisitionEngine(left, right, FrameHub(), sink=sink.append) + engine.run(max_ticks=4) + assert [p.frame_index for p in sink] == [0, 1, 2, 3] + + +def test_run_loop_exits_when_stop_called(): + left = FakeCamera(_imgs(1000)) + right = FakeCamera(_imgs(1000, base=1)) + engine = AcquisitionEngine(left, right, FrameHub()) + # stop before running; loop should not produce more than a couple of ticks + engine.stop() + engine.run(max_ticks=1000) + assert engine.ticks_completed <= 1 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_acquisition.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.acquisition'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/acquisition.py`: + +```python +import threading +import time +from typing import Callable, Optional + +from core.cameras import Camera +from core.frames import AcquiredPair, FrameHub + +Sink = Callable[[AcquiredPair], None] + + +class AcquisitionEngine: + """Single loop that owns both cameras. Grabs both per tick, assigns a shared monotonic + frame_index, publishes the latest frame of each to the hub for display/tracking, and + forwards the paired result to an optional sink (the recorder). This is the only place that + reads from the cameras, so the two streams cannot drift relative to each other.""" + + LEFT = "left" + RIGHT = "right" + + def __init__(self, left: Camera, right: Camera, hub: FrameHub, + sink: Optional[Sink] = None) -> None: + self._left = left + self._right = right + self._hub = hub + self._sink = sink + self._next_index = 0 + self._stop = threading.Event() + self.ticks_completed = 0 + + def run_once(self) -> AcquiredPair: + index = self._next_index + self._next_index += 1 + ts = time.monotonic() + + left_frame = self._left.read() + right_frame = self._right.read() + + if left_frame is not None: + self._hub.publish(self.LEFT, left_frame) + if right_frame is not None: + self._hub.publish(self.RIGHT, right_frame) + + pair = AcquiredPair(frame_index=index, capture_ts=ts, + left=left_frame, right=right_frame) + if self._sink is not None: + self._sink(pair) + self.ticks_completed += 1 + return pair + + def run(self, max_ticks: Optional[int] = None) -> None: + produced = 0 + while not self._stop.is_set(): + if max_ticks is not None and produced >= max_ticks: + break + if not (self._left.is_open() and self._right.is_open()): + break + self.run_once() + produced += 1 + + def stop(self) -> None: + self._stop.set() +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_acquisition.py` +Expected: PASS (8 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/acquisition.py WormSpy/backend/code/tests/test_acquisition.py +git commit -m "feat: add AcquisitionEngine with shared monotonic frame index" +``` + +--- + +### Task 6: Tracking helpers and the Tracker thread + +**Files:** +- Create: `WormSpy/backend/code/core/tracking.py` +- Test: `WormSpy/backend/code/tests/test_tracking.py` +- Reference (copy logic from): `app.py:544-613` (`simpleToCenter`, `find_worm_cms`) + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_tracking.py`: + +```python +import numpy as np +from core.frames import FrameHub, Frame +from core.stage import FakeStage +from core.tracking import find_worm_cms, simple_to_center, Tracker + + +def test_find_worm_cms_returns_centroid_of_largest_blob(): + # 40x40 frame, a bright square blob centered near (10, 20) in (row, col) + frame = np.zeros((40, 40), np.uint8) + frame[8:13, 18:23] = 255 # rows 8-12, cols 18-22 -> centroid ~ (10, 20) + x, y = find_worm_cms(frame, factor=1, initial_coords=(20, 20)) + # returned as (x=col, y=row) + assert abs(x - 20) <= 1 + assert abs(y - 10) <= 1 + + +def test_find_worm_cms_applies_downsample_factor(): + frame = np.zeros((20, 20), np.uint8) + frame[9:11, 9:11] = 255 # centroid ~ (9.5, 9.5) + x, y = find_worm_cms(frame, factor=2, initial_coords=(10, 10)) + assert abs(x - 19) <= 2 # ~9.5 * 2 + assert abs(y - 19) <= 2 + + +def test_find_worm_cms_falls_back_to_initial_when_empty(): + frame = np.zeros((10, 10), np.uint8) + assert find_worm_cms(frame, factor=1, initial_coords=(5, 5)) == (5, 5) + + +def test_simple_to_center_centered_worm_needs_no_move(): + # worm exactly at center -> zero move + mx, my = simple_to_center(960, 600, resolution=(1920, 1200), + total_mm_x=1.92, total_mm_y=1.2, + orient_x=1, orient_y=-1) + assert abs(mx) < 1e-9 + assert abs(my) < 1e-9 + + +def test_simple_to_center_respects_orientation_sign(): + # worm to the right of center -> positive-ish move scaled by orient_x + mx, _ = simple_to_center(1920, 600, resolution=(1920, 1200), + total_mm_x=1.92, total_mm_y=1.2, + orient_x=1, orient_y=-1) + assert mx > 0 + mx_inv, _ = simple_to_center(1920, 600, resolution=(1920, 1200), + total_mm_x=1.92, total_mm_y=1.2, + orient_x=-1, orient_y=-1) + assert mx_inv < 0 + + +def test_tracker_step_does_nothing_when_disabled(): + hub = FrameHub() + hub.publish("left", Frame(np.zeros((40, 40), np.uint8), 0, 0.0)) + stage = FakeStage(limits={"x": (0.0, 1e9), "y": (0.0, 1e9)}, + positions={"x": 100.0, "y": 100.0}) + tracker = Tracker(hub, stage, enabled=lambda: False, mode=lambda: 0, + compute=lambda frame, mode: (5, 5)) + tracker.step() + assert stage.get_position("x") == 100.0 # unchanged + + +def test_tracker_step_commands_stage_when_enabled(): + hub = FrameHub() + hub.publish("left", Frame(np.zeros((40, 40), np.uint8), 0, 0.0)) + stage = FakeStage(limits={"x": (-1e9, 1e9), "y": (-1e9, 1e9)}, + positions={"x": 0.0, "y": 0.0}) + moves = [] + tracker = Tracker(hub, stage, enabled=lambda: True, mode=lambda: 0, + compute=lambda frame, mode: (123.0, -45.0), + on_move=lambda dx, dy, hit: moves.append((dx, dy))) + tracker.step() + assert moves == [(123.0, -45.0)] + assert stage.get_position("x") == 123.0 + assert stage.get_position("y") == -45.0 + + +def test_tracker_step_skips_when_no_frame_available(): + hub = FrameHub() # nothing published + stage = FakeStage(limits={"x": (0.0, 10.0), "y": (0.0, 10.0)}, + positions={"x": 1.0, "y": 1.0}) + tracker = Tracker(hub, stage, enabled=lambda: True, mode=lambda: 0, + compute=lambda frame, mode: (5, 5)) + tracker.step() # must not raise + assert stage.get_position("x") == 1.0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_tracking.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.tracking'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/tracking.py`: + +```python +"""Pure tracking helpers (moved from app.py) and the Tracker thread. + +The Tracker reads the latest brightfield frame from the hub, computes a target stage move via +an injected `compute` callback, and commands the stage. Compute is injected so the heavy/slow +algorithm (e.g. DeepLabCut) can run here without ever throttling the AcquisitionEngine.""" + +import threading +import time +from typing import Callable, Optional, Tuple + +import numpy as np +from skimage.measure import label, regionprops + +from core.frames import FrameHub +from core.stage import Stage + + +def find_worm_cms(processed_frame: np.ndarray, factor: int, + initial_coords: Tuple[float, float]) -> Tuple[float, float]: + """Centroid (x=col, y=row) of the largest connected region, rescaled by `factor`. + Falls back to initial_coords when no region is found.""" + labeled = label(processed_frame) + regions = regionprops(labeled) + regions_by_area = sorted(regions, key=lambda r: r.area, reverse=True) + if regions_by_area: + coords = regions_by_area[0].centroid # (row, col) + else: + return initial_coords + coords = (round(coords[1]), round(coords[0])) # -> (x=col, y=row) + return (coords[0] * factor, coords[1] * factor) + + +def simple_to_center(centroid_x: float, centroid_y: float, + resolution: Tuple[float, float], + total_mm_x: float, total_mm_y: float, + orient_x: int, orient_y: int) -> Tuple[float, float]: + """Millimetres the stage must move to bring (centroid_x, centroid_y) to frame center.""" + percent_x = float(centroid_x) / float(resolution[0]) + percent_y = float(centroid_y) / float(resolution[1]) + millis_x = percent_x * total_mm_x + millis_y = percent_y * total_mm_y + move_x = orient_x * (millis_x - total_mm_x / 2) + move_y = orient_y * (millis_y - total_mm_y / 2) + return move_x, move_y + + +# Callback types +EnabledFn = Callable[[], bool] +ModeFn = Callable[[], int] +ComputeFn = Callable[[np.ndarray, int], Tuple[float, float]] # (frame, mode) -> (dx, dy) native +OnMoveFn = Callable[[float, float, bool], None] + + +class Tracker: + """Runs in its own thread. Best-effort consumer of the latest brightfield frame.""" + + def __init__(self, hub: FrameHub, stage: Stage, + enabled: EnabledFn, mode: ModeFn, compute: ComputeFn, + on_move: Optional[OnMoveFn] = None, + stream: str = "left", interval_s: float = 0.0) -> None: + self._hub = hub + self._stage = stage + self._enabled = enabled + self._mode = mode + self._compute = compute + self._on_move = on_move + self._stream = stream + self._interval_s = interval_s + self._stop = threading.Event() + + def step(self) -> None: + if not self._enabled(): + return + frame = self._hub.latest(self._stream) + if frame is None: + return + dx, dy = self._compute(frame.image, self._mode()) + hit_x = self._stage.move_relative("x", dx) + hit_y = self._stage.move_relative("y", dy) + if self._on_move is not None: + self._on_move(dx, dy, hit_x or hit_y) + + def run(self) -> None: + while not self._stop.is_set(): + self.step() + if self._interval_s: + time.sleep(self._interval_s) + + def stop(self) -> None: + self._stop.set() +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_tracking.py` +Expected: PASS (9 passed) + +- [ ] **Step 5: Run the whole suite to confirm nothing regressed** + +Run: `cd WormSpy/backend/code && python -m pytest` +Expected: PASS (all tests across the 5 files) + +- [ ] **Step 6: Commit** + +```bash +git add WormSpy/backend/code/core/tracking.py WormSpy/backend/code/tests/test_tracking.py +git commit -m "feat: add pure tracking helpers and Tracker thread" +``` + +--- + +### Task 7: ZaberStage adapter and connect-once factory + +**Files:** +- Create: `WormSpy/backend/code/core/stage_zaber.py` +- Test: `WormSpy/backend/code/tests/test_stage.py` (extend) +- Reference (copy logic from): `app.py:127-142` (connection/detect), `app.py:573-576` (move command format), `app.py:457-462` (limits) + +> **Note:** `ZaberStage` itself talks to hardware and is verified manually on the rig (Step 5). Only the connection *factory's fallback* is unit-tested, by injecting a connector that raises. + +- [ ] **Step 1: Write the failing test (factory fallback only)** + +Append to `tests/test_stage.py`: + +```python +from core.stage_zaber import connect_stage + + +def test_connect_stage_returns_null_stage_when_connection_fails(): + def boom(*_args, **_kwargs): + raise RuntimeError("no serial port") + stage = connect_stage(xy_port="COM6", z_port="COM3", connector=boom) + assert stage.is_connected() is False # NullStage + + +def test_connect_stage_uses_injected_connector_result(): + sentinel = FakeStage(limits={"x": (0.0, 1.0), "y": (0.0, 1.0), "z": (0.0, 1.0)}, + positions={"x": 0.0, "y": 0.0, "z": 0.0}) + stage = connect_stage(xy_port="COM6", z_port="COM3", connector=lambda **_: sentinel) + assert stage is sentinel +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_stage.py -k connect_stage` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.stage_zaber'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/stage_zaber.py`: + +```python +"""Zaber hardware adapter. Imports zaber_motion lazily so tests never load the SDK.""" + +from typing import Callable, Optional, Tuple + +from core.stage import Stage, NullStage, clamp_native + + +class ZaberStage(Stage): + """Wraps three Zaber axes (x, y on the XY connection; z on the Z connection). + + Connections are opened once at construction and held for the process lifetime.""" + + def __init__(self, xy_port: str, z_port: str) -> None: + from zaber_motion import Library # lazy import + from zaber_motion.ascii import Connection + Library.enable_device_db_store() + + self._xy_conn = Connection.open_serial_port(xy_port) + self._z_conn = Connection.open_serial_port(z_port) + self._xy_conn.enable_alerts() + self._z_conn.enable_alerts() + + horizontal = self._xy_conn.detect_devices() + vertical = self._z_conn.detect_devices() + self._axes = { + "x": horizontal[0].get_axis(1), + "y": horizontal[1].get_axis(1), + "z": vertical[0].get_axis(1), + } + self._limits = { + name: (float(axis.settings.get("limit.min")), + float(axis.settings.get("limit.max"))) + for name, axis in self._axes.items() + } + + def is_connected(self) -> bool: + return True + + def get_position(self, axis: str) -> float: + from zaber_motion import Units + return float(self._axes[axis].get_position(unit=Units.NATIVE)) + + def limits(self, axis: str) -> Tuple[float, float]: + return self._limits[axis] + + def move_relative(self, axis: str, native: float) -> bool: + lo, hi = self._limits[axis] + current = self.get_position(axis) + target, hit = clamp_native(current + native, lo, hi) + delta = int(target - current) + if delta != 0: + self._axes[axis].generic_command_no_response(f"move rel {delta} 10000 5") + return hit + + def move_absolute(self, axis: str, native: float) -> bool: + from zaber_motion import Units + lo, hi = self._limits[axis] + target, hit = clamp_native(native, lo, hi) + self._axes[axis].move_absolute(target, unit=Units.NATIVE, wait_until_idle=False) + return hit + + +def connect_stage(xy_port: str, z_port: str, + connector: Optional[Callable[..., Stage]] = None) -> Stage: + """Open the stage once at startup. On any failure, return a NullStage so the server still + runs (e.g. for UI/dev work with no hardware). `connector` is injectable for tests.""" + factory = connector if connector is not None else (lambda **kw: ZaberStage(**kw)) + try: + return factory(xy_port=xy_port, z_port=z_port) + except Exception as exc: # noqa: BLE001 - degrade gracefully on any hardware/SDK error + print(f"connect_stage: no stage hardware ({exc!r}); using NullStage.") + return NullStage() +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd WormSpy/backend/code && python -m pytest tests/test_stage.py -k connect_stage` +Expected: PASS (2 passed) + +- [ ] **Step 5: Manual hardware verification (on the rig only)** + +On the rig with Zaber connected and ZaberLauncher running, in a Python shell from `WormSpy/backend/code`: + +```python +from core.stage_zaber import connect_stage +s = connect_stage("COM6", "COM3") +print(s.is_connected(), s.limits("x")) +s.move_relative("x", 1000) # observe the XY stage step; confirm no error +``` + +Expected: `True` and a `(min, max)` tuple; the stage moves a small step. + +- [ ] **Step 6: Commit** + +```bash +git add WormSpy/backend/code/core/stage_zaber.py WormSpy/backend/code/tests/test_stage.py +git commit -m "feat: add ZaberStage adapter with graceful connect-once fallback" +``` + +--- + +### Task 8: SpinnakerCamera adapter (camera-as-clock) + +**Files:** +- Create: `WormSpy/backend/code/core/cameras_spinnaker.py` +- Reference (copy logic from): `app.py:108-109` (open + resolution), `test_segmentation.py:6-13` (EasyPySpin read + uint8 normalize) + +> **Note:** This adapter talks to hardware and is verified manually (Step 4). It configures the camera to be the clock: fixed frame rate + fixed exposure, so every recorded frame shares an exposure and the rate does not wobble with software load (ADR 0001 decision #2). + +- [ ] **Step 1: Write the adapter** + +Create `core/cameras_spinnaker.py`: + +```python +"""FLIR/Spinnaker camera adapter via EasyPySpin. Imports the SDK lazily so tests don't load it. + +Sets the camera as the acquisition clock: a fixed AcquisitionFrameRate and a fixed exposure, +with auto-exposure/auto-gain off. EasyPySpin exposes these through cv2-style properties; the +device frame id is read from the grabbed image metadata when available, else -1.""" + +import time +from typing import Optional + +import cv2 +import numpy as np + +from core.cameras import Camera +from core.frames import Frame + + +class SpinnakerCamera(Camera): + def __init__(self, identifier, frame_rate: float, exposure_us: float) -> None: + import EasyPySpin # lazy import + self._cap = EasyPySpin.VideoCapture(identifier) + if not self._cap.isOpened(): + print(f"SpinnakerCamera: could not open camera {identifier!r}") + return + # Camera is the clock: disable auto, pin rate + exposure. + self._cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 0) # manual exposure + self._cap.set(cv2.CAP_PROP_EXPOSURE, exposure_us) + self._cap.set(cv2.CAP_PROP_FPS, frame_rate) + self.width = self._cap.get(cv2.CAP_PROP_FRAME_WIDTH) + self.height = self._cap.get(cv2.CAP_PROP_FRAME_HEIGHT) + + def read(self) -> Optional[Frame]: + ok, image = self._cap.read() + ts = time.monotonic() + if not ok or image is None: + return None + if image.dtype != np.uint8: + # Preserve raw for recording elsewhere; this path is display-oriented in Phase 1. + image = cv2.normalize(image, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U) + if len(image.shape) == 3: + image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) + frame_id = int(self._cap.get(cv2.CAP_PROP_POS_FRAMES)) + return Frame(image=image, device_frame_id=frame_id, capture_ts=ts) + + def is_open(self) -> bool: + return self._cap is not None and self._cap.isOpened() + + def release(self) -> None: + if self._cap is not None: + self._cap.release() +``` + +- [ ] **Step 2: Import-smoke check (dev machine, no camera)** + +Run: `cd WormSpy/backend/code && python -c "import ast; ast.parse(open('core/cameras_spinnaker.py').read()); print('parse ok')"` +Expected: `parse ok` (syntax valid; the SDK is not imported because the lazy import is inside `__init__`). + +- [ ] **Step 3: Confirm the rest of the suite is unaffected** + +Run: `cd WormSpy/backend/code && python -m pytest` +Expected: PASS (no new tests; nothing regressed) + +- [ ] **Step 4: Manual hardware verification (on the rig only)** + +With a FLIR camera connected, from `WormSpy/backend/code`: + +```python +from core.cameras_spinnaker import SpinnakerCamera +cam = SpinnakerCamera(0, frame_rate=10.0, exposure_us=10000) +f = cam.read() +print(cam.is_open(), None if f is None else (f.image.shape, f.device_frame_id)) +cam.release() +``` + +Expected: `True` and a `(shape, frame_id)` tuple printed. + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/cameras_spinnaker.py +git commit -m "feat: add SpinnakerCamera adapter configured as the acquisition clock" +``` + +--- + +### Task 9: Wire the engine into app.py; serve display from the hub + +**Files:** +- Modify: `WormSpy/backend/code/app.py` + - Replace per-request camera/motor setup in `video_feed` (`app.py:104-237`) and `video_feed_fluorescent` (`app.py:240-313`). + - Add startup construction near `app = Flask(...)` (`app.py:24-27`). + - Repoint `move_to_center` (`app.py:454-463`) and the manual controller (`app.py:656-749`) at the shared `STAGE`. + +> **Note:** This task is hardware-wired and validated by launching the app on the rig (Step 5). It removes acquisition from the HTTP handlers (the root cause of loop drift and motor-on-disconnect). Recording and Hold Focus are intentionally left on their existing code paths for now (Phases 2–3); the engine `sink` is wired but set to a no-op placeholder so behaviour is unchanged until Phase 2. + +- [ ] **Step 1: Add startup wiring after the Flask app is created** + +In `app.py`, immediately after the `CORS(...)` block (after `app.py:28`), add: + +```python +from core.frames import FrameHub +from core.cameras_spinnaker import SpinnakerCamera +from core.stage_zaber import connect_stage +from core.acquisition import AcquisitionEngine +from core.tracking import Tracker, find_worm_cms, simple_to_center +import threading as _threading + +# Shared singletons constructed once at startup (replaces per-request setup). +HUB = FrameHub() +STAGE = connect_stage(XYmotorport, Zmotorport) # NullStage if no hardware +ENGINE = None # set when the live feed starts (cameras chosen in the UI) +ACQ_THREAD = None + +def _start_engine(left_id, right_id): + global ENGINE, ACQ_THREAD + left = SpinnakerCamera(left_id, frame_rate=FPS, exposure_us=10000) + right = SpinnakerCamera(right_id, frame_rate=FPS, exposure_us=10000) + ENGINE = AcquisitionEngine(left, right, HUB, sink=lambda pair: None) # recorder: Phase 2 + ACQ_THREAD = _threading.Thread(target=ENGINE.run, daemon=True) + ACQ_THREAD.start() +``` + +- [ ] **Step 2: Start the engine from `/camera_settings` instead of opening cameras per request** + +Replace the body of `camera_settings` (`app.py:404-411`) with: + +```python +@cross_origin() +@app.route("/camera_settings", methods=['POST']) +def camera_settings(): + leftCam = request.json['leftCam'] + rightCam = request.json['rightCam'] + _start_engine(leftCam, rightCam) + return jsonify({"message": "Engine started"}) +``` + +- [ ] **Step 3: Serve both display feeds from the hub** + +Replace `video_feed` (`app.py:104-237`) and `video_feed_fluorescent` (`app.py:240-313`) with thin hub readers: + +```python +def _mjpeg_from_hub(stream): + def gen(): + while True: + frame = HUB.latest(stream) + if frame is None: + time.sleep(0.03) + continue + ok, buf = cv2.imencode('.jpg', frame.image) # jpeg matches the declared mimetype + if ok: + yield (b'--frame\r\n' + b'Content-Type: image/jpeg\r\n\r\n' + buf.tobytes() + b'\r\n') + time.sleep(0.03) + return Response(gen(), mimetype='multipart/x-mixed-replace; boundary=frame') + +@cross_origin() +@app.route('/video_feed') +def video_feed(): + return _mjpeg_from_hub("left") + +@cross_origin() +@app.route('/video_feed_fluorescent') +def video_feed_fluorescent(): + return _mjpeg_from_hub("right") +``` + +- [ ] **Step 4: Repoint `move_to_center` at the shared STAGE** + +Replace `move_to_center` (`app.py:454-463`) with: + +```python +@cross_origin() +@app.route("/move_to_center", methods=['POST']) +def move_to_center(): + lo_x, hi_x = STAGE.limits("x") + lo_y, hi_y = STAGE.limits("y") + STAGE.move_absolute("x", (lo_x + hi_x) / 2) + STAGE.move_absolute("y", (lo_y + hi_y) / 2) + return jsonify({"message": "Moved to center"}) +``` + +- [ ] **Step 5: Manual end-to-end verification (on the rig)** + +1. Start the app: `cd WormSpy/backend/code && python app.py` +2. Visit `localhost:5000`, pick the two cameras, Start Live Feed. +3. Confirm: both feeds display; refreshing the browser does NOT stop acquisition (the terminal shows the engine still running); "Center" moves the stage even right after a refresh; closing the browser tab does not kill the motor connection. + +Expected: both feeds live, stage controllable independent of the browser, no `NameError` in the terminal. + +- [ ] **Step 6: Run the unit suite once more** + +Run: `cd WormSpy/backend/code && python -m pytest` +Expected: PASS (all tests) + +- [ ] **Step 7: Commit** + +```bash +git add WormSpy/backend/code/app.py +git commit -m "refactor: own acquisition at startup; serve display from FrameHub (ADR 0001)" +``` + +--- + +## Self-Review + +**Spec coverage (ADR 0001 decisions):** +1. *Single acquisition loop owning both cameras with shared frame_index* → Task 5 (engine) + Task 9 (wired at startup). ✓ +2. *Camera as the clock (fixed rate + fixed exposure), frame IDs logged* → Task 8 (config) + `device_frame_id` carried through Tasks 2/5 (`AcquiredPair`). Note: writing IDs to the CSV is Phase 2 (ADR 0003); Phase 1 only *preserves* them. ✓ (within scope) +3. *Zaber connection opened once at startup, degrade gracefully* → Task 7 (`connect_stage` → NullStage) + Task 9 (constructed at startup). ✓ +4. *Tracking in its own thread for all modes* → Task 6 (`Tracker`, compute injected so DLC runs off the acquisition loop). ✓ + +**Out-of-scope, deferred deliberately (called out in headers):** new recording format + CSV spine + codec (Phase 2), Hold Focus (Phase 3), `/status` + UI (Phase 4). The engine `sink` is a no-op placeholder so recording behaviour is unchanged until Phase 2. + +**Placeholder scan:** No "TBD"/"add error handling"/"similar to" placeholders; every code step contains complete code. Manual-verification steps (7.5, 8.4, 9.5) are explicit hardware procedures, not code placeholders. + +**Type consistency:** `Frame(image, device_frame_id, capture_ts)` and `AcquiredPair(frame_index, capture_ts, left, right)` used identically across Tasks 2/5/6/8. `Stage.move_relative/move_absolute` return `bool` (limit-hit) consistently across Tasks 4/7 and consumed in Task 6. `find_worm_cms(frame, factor, initial_coords)` and `simple_to_center(...)` signatures match between Task 6 definition and its tests. `connect_stage(xy_port, z_port, connector=None)` matches between Task 7 definition and tests. + +**Known follow-ups for later phases (not gaps in this plan):** the `factor=2` downsample, the per-mode `compute` implementation (thresholding/fluorescent/DLC) wiring into `Tracker`, and `move_to_center` using true device midpoints all land when Phases 2–3 replace the remaining legacy code paths. From 7a600d877896c7a5150c1e3b90a8c15930498134 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 09:48:46 -0400 Subject: [PATCH 02/13] test: add pytest harness and core package skeleton Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/core/__init__.py | 1 + WormSpy/backend/code/pytest.ini | 4 ++++ WormSpy/backend/code/requirements-dev.txt | 2 ++ WormSpy/backend/code/tests/conftest.py | 5 +++++ 4 files changed, 12 insertions(+) create mode 100644 WormSpy/backend/code/core/__init__.py create mode 100644 WormSpy/backend/code/pytest.ini create mode 100644 WormSpy/backend/code/requirements-dev.txt create mode 100644 WormSpy/backend/code/tests/conftest.py diff --git a/WormSpy/backend/code/core/__init__.py b/WormSpy/backend/code/core/__init__.py new file mode 100644 index 0000000..079b4bc --- /dev/null +++ b/WormSpy/backend/code/core/__init__.py @@ -0,0 +1 @@ +"""WormsPy acquisition core: hardware-independent acquisition, tracking, and stage control.""" diff --git a/WormSpy/backend/code/pytest.ini b/WormSpy/backend/code/pytest.ini new file mode 100644 index 0000000..99c4474 --- /dev/null +++ b/WormSpy/backend/code/pytest.ini @@ -0,0 +1,4 @@ +[pytest] +testpaths = tests +python_files = test_*.py +addopts = -v diff --git a/WormSpy/backend/code/requirements-dev.txt b/WormSpy/backend/code/requirements-dev.txt new file mode 100644 index 0000000..5d107fb --- /dev/null +++ b/WormSpy/backend/code/requirements-dev.txt @@ -0,0 +1,2 @@ +pytest>=7.0 +numpy>=1.20 diff --git a/WormSpy/backend/code/tests/conftest.py b/WormSpy/backend/code/tests/conftest.py new file mode 100644 index 0000000..fa5c75a --- /dev/null +++ b/WormSpy/backend/code/tests/conftest.py @@ -0,0 +1,5 @@ +import os +import sys + +# Ensure `import core...` resolves when pytest is run from WormSpy/backend/code +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) From c7f9ded47e77fbaa8ffa6a1820a682b99b2008d7 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 09:50:52 -0400 Subject: [PATCH 03/13] feat: add Frame/AcquiredPair types and thread-safe FrameHub Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/core/frames.py | 38 +++++++++++++++++ WormSpy/backend/code/tests/test_frames.py | 51 +++++++++++++++++++++++ 2 files changed, 89 insertions(+) create mode 100644 WormSpy/backend/code/core/frames.py create mode 100644 WormSpy/backend/code/tests/test_frames.py diff --git a/WormSpy/backend/code/core/frames.py b/WormSpy/backend/code/core/frames.py new file mode 100644 index 0000000..ed616af --- /dev/null +++ b/WormSpy/backend/code/core/frames.py @@ -0,0 +1,38 @@ +import threading +from dataclasses import dataclass +from typing import Dict, Optional + +import numpy as np + + +@dataclass +class Frame: + """A single grabbed image plus the metadata needed to align it offline.""" + image: np.ndarray + device_frame_id: int # camera-provided frame id; -1 if the camera cannot supply one + capture_ts: float # time.monotonic() captured as close to the grab as possible + + +@dataclass +class AcquiredPair: + """One acquisition tick: a brightfield and a fluorescence frame sharing an index.""" + frame_index: int # monotonic counter assigned by the engine, contiguous, never reused + capture_ts: float # engine timestamp for the tick (time.monotonic()) + left: Optional[Frame] # brightfield/behaviour; None if that grab failed this tick + right: Optional[Frame] # fluorescence/calcium; None if that grab failed this tick + + +class FrameHub: + """Thread-safe store of the latest frame per named stream, for display and tracking.""" + + def __init__(self) -> None: + self._lock = threading.Lock() + self._latest: Dict[str, Frame] = {} + + def publish(self, name: str, frame: Frame) -> None: + with self._lock: + self._latest[name] = frame + + def latest(self, name: str) -> Optional[Frame]: + with self._lock: + return self._latest.get(name) diff --git a/WormSpy/backend/code/tests/test_frames.py b/WormSpy/backend/code/tests/test_frames.py new file mode 100644 index 0000000..eec78aa --- /dev/null +++ b/WormSpy/backend/code/tests/test_frames.py @@ -0,0 +1,51 @@ +import threading +import numpy as np +from core.frames import Frame, AcquiredPair, FrameHub + + +def _frame(val, fid=0, ts=0.0): + img = np.full((4, 4), val, dtype=np.uint8) + return Frame(image=img, device_frame_id=fid, capture_ts=ts) + + +def test_hub_returns_none_before_first_publish(): + hub = FrameHub() + assert hub.latest("left") is None + + +def test_hub_returns_most_recent_frame(): + hub = FrameHub() + hub.publish("left", _frame(1, fid=1)) + hub.publish("left", _frame(2, fid=2)) + latest = hub.latest("left") + assert latest.device_frame_id == 2 + assert int(latest.image[0, 0]) == 2 + + +def test_hub_keeps_streams_independent(): + hub = FrameHub() + hub.publish("left", _frame(1)) + hub.publish("right", _frame(9)) + assert int(hub.latest("left").image[0, 0]) == 1 + assert int(hub.latest("right").image[0, 0]) == 9 + + +def test_hub_is_thread_safe_under_concurrent_writes(): + hub = FrameHub() + def writer(start): + for i in range(start, start + 500): + hub.publish("left", _frame(i % 256, fid=i)) + threads = [threading.Thread(target=writer, args=(s,)) for s in (0, 1000)] + for t in threads: + t.start() + for t in threads: + t.join() + assert hub.latest("left") is not None + + +def test_acquired_pair_fields(): + pair = AcquiredPair(frame_index=7, capture_ts=1.5, left=_frame(1), right=_frame(2)) + assert pair.frame_index == 7 + assert pair.capture_ts == 1.5 + assert pair.left.device_frame_id == 0 + assert pair.right is not None From 176dce26e0af9a79230bfdecc86919dabf7c88bb Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 09:52:36 -0400 Subject: [PATCH 04/13] feat: add Camera interface and FakeCamera test double Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/core/cameras.py | 53 ++++++++++++++++++++++ WormSpy/backend/code/tests/test_cameras.py | 46 +++++++++++++++++++ 2 files changed, 99 insertions(+) create mode 100644 WormSpy/backend/code/core/cameras.py create mode 100644 WormSpy/backend/code/tests/test_cameras.py diff --git a/WormSpy/backend/code/core/cameras.py b/WormSpy/backend/code/core/cameras.py new file mode 100644 index 0000000..0c09ea8 --- /dev/null +++ b/WormSpy/backend/code/core/cameras.py @@ -0,0 +1,53 @@ +import time +from abc import ABC, abstractmethod +from typing import Iterable, List, Optional, Set + +import numpy as np + +from core.frames import Frame + + +class Camera(ABC): + """Acquisition interface. Adapters wrap real SDKs; FakeCamera is the test double.""" + + @abstractmethod + def read(self) -> Optional[Frame]: + """Grab the next frame, or return None if the grab failed / stream ended.""" + + @abstractmethod + def is_open(self) -> bool: + ... + + @abstractmethod + def release(self) -> None: + ... + + +class FakeCamera(Camera): + """Deterministic camera for tests. Emits a scripted list of images.""" + + def __init__(self, images: Iterable[np.ndarray], + frame_ids: Optional[List[int]] = None, + fail_on: Optional[Set[int]] = None) -> None: + self._images: List[np.ndarray] = list(images) + self._frame_ids = frame_ids if frame_ids is not None else list(range(len(self._images))) + self._fail_on = fail_on or set() + self._pos = 0 + self._open = True + + def read(self) -> Optional[Frame]: + if not self._open or self._pos >= len(self._images): + return None + idx = self._pos + self._pos += 1 + if idx in self._fail_on: + return None + return Frame(image=self._images[idx], + device_frame_id=self._frame_ids[idx], + capture_ts=time.monotonic()) + + def is_open(self) -> bool: + return self._open + + def release(self) -> None: + self._open = False diff --git a/WormSpy/backend/code/tests/test_cameras.py b/WormSpy/backend/code/tests/test_cameras.py new file mode 100644 index 0000000..e9225fa --- /dev/null +++ b/WormSpy/backend/code/tests/test_cameras.py @@ -0,0 +1,46 @@ +import numpy as np +from core.cameras import Camera, FakeCamera + + +def test_fake_camera_yields_scripted_frames_in_order(): + cam = FakeCamera(images=[np.full((2, 2), 1, np.uint8), + np.full((2, 2), 2, np.uint8)]) + f0 = cam.read() + f1 = cam.read() + assert int(f0.image[0, 0]) == 1 + assert int(f1.image[0, 0]) == 2 + + +def test_fake_camera_assigns_incrementing_device_frame_ids(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)] * 3) + ids = [cam.read().device_frame_id for _ in range(3)] + assert ids == [0, 1, 2] + + +def test_fake_camera_can_simulate_dropped_frames(): + # device_frame_ids 0,1,3 -> a gap (2 dropped) that downstream code can detect + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)] * 3, frame_ids=[0, 1, 3]) + ids = [cam.read().device_frame_id for _ in range(3)] + assert ids == [0, 1, 3] + + +def test_fake_camera_returns_none_when_exhausted(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)]) + assert cam.read() is not None + assert cam.read() is None + + +def test_fake_camera_read_failure_returns_none(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)], fail_on={0}) + assert cam.read() is None + + +def test_fake_camera_is_open_until_released(): + cam = FakeCamera(images=[np.zeros((2, 2), np.uint8)]) + assert cam.is_open() is True + cam.release() + assert cam.is_open() is False + + +def test_fake_camera_satisfies_camera_interface(): + assert issubclass(FakeCamera, Camera) From c4d017388f7bff5aa1a050feec7a01cbca85e182 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 09:54:24 -0400 Subject: [PATCH 05/13] feat: add Stage interface, clamp helper, Null/Fake stages Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/core/stage.py | 84 ++++++++++++++++++++++++ WormSpy/backend/code/tests/test_stage.py | 50 ++++++++++++++ 2 files changed, 134 insertions(+) create mode 100644 WormSpy/backend/code/core/stage.py create mode 100644 WormSpy/backend/code/tests/test_stage.py diff --git a/WormSpy/backend/code/core/stage.py b/WormSpy/backend/code/core/stage.py new file mode 100644 index 0000000..748de66 --- /dev/null +++ b/WormSpy/backend/code/core/stage.py @@ -0,0 +1,84 @@ +from abc import ABC, abstractmethod +from typing import Dict, Tuple + + +def clamp_native(value: float, minimum: float, maximum: float) -> Tuple[float, bool]: + """Clamp value into [minimum, maximum]. Returns (clamped_value, limit_was_hit).""" + if value > maximum: + return maximum, True + if value < minimum: + return minimum, True + return value, False + + +class Stage(ABC): + """XYZ stage interface in native (microstep) units. Adapters wrap real hardware.""" + + @abstractmethod + def is_connected(self) -> bool: + ... + + @abstractmethod + def get_position(self, axis: str) -> float: + ... + + @abstractmethod + def limits(self, axis: str) -> Tuple[float, float]: + ... + + @abstractmethod + def move_relative(self, axis: str, native: float) -> bool: + """Move axis by `native` microsteps, clamped to limits. Returns True if a limit was hit.""" + + @abstractmethod + def move_absolute(self, axis: str, native: float) -> bool: + """Move axis to absolute `native` microsteps, clamped to limits. Returns True if hit.""" + + +class NullStage(Stage): + """Used when no hardware is present so the app still runs. All moves are no-ops.""" + + def is_connected(self) -> bool: + return False + + def get_position(self, axis: str) -> float: + return 0.0 + + def limits(self, axis: str) -> Tuple[float, float]: + return (0.0, 0.0) + + def move_relative(self, axis: str, native: float) -> bool: + return False + + def move_absolute(self, axis: str, native: float) -> bool: + return False + + +class FakeStage(Stage): + """In-memory stage for tests. Honors limits via clamp_native.""" + + def __init__(self, limits: Dict[str, Tuple[float, float]], + positions: Dict[str, float]) -> None: + self._limits = dict(limits) + self._pos = dict(positions) + + def is_connected(self) -> bool: + return True + + def get_position(self, axis: str) -> float: + return self._pos[axis] + + def limits(self, axis: str) -> Tuple[float, float]: + return self._limits[axis] + + def move_relative(self, axis: str, native: float) -> bool: + lo, hi = self._limits[axis] + target, hit = clamp_native(self._pos[axis] + native, lo, hi) + self._pos[axis] = target + return hit + + def move_absolute(self, axis: str, native: float) -> bool: + lo, hi = self._limits[axis] + target, hit = clamp_native(native, lo, hi) + self._pos[axis] = target + return hit diff --git a/WormSpy/backend/code/tests/test_stage.py b/WormSpy/backend/code/tests/test_stage.py new file mode 100644 index 0000000..edf575d --- /dev/null +++ b/WormSpy/backend/code/tests/test_stage.py @@ -0,0 +1,50 @@ +import pytest +from core.stage import Stage, NullStage, FakeStage, clamp_native + + +def test_clamp_within_bounds_is_unchanged(): + value, hit = clamp_native(50.0, 0.0, 100.0) + assert value == 50.0 + assert hit is False + + +def test_clamp_above_max_pins_to_max_and_flags(): + value, hit = clamp_native(150.0, 0.0, 100.0) + assert value == 100.0 + assert hit is True + + +def test_clamp_below_min_pins_to_min_and_flags(): + value, hit = clamp_native(-5.0, 0.0, 100.0) + assert value == 0.0 + assert hit is True + + +def test_null_stage_is_not_connected(): + stage = NullStage() + assert stage.is_connected() is False + + +def test_null_stage_moves_are_noops(): + stage = NullStage() + stage.move_relative("x", 100.0) + assert stage.get_position("x") == 0.0 + + +def test_fake_stage_tracks_relative_moves_and_clamps(): + stage = FakeStage(limits={"x": (0.0, 100.0)}, positions={"x": 90.0}) + hit = stage.move_relative("x", 50.0) # 90 + 50 = 140 -> clamp to 100 + assert stage.get_position("x") == 100.0 + assert hit is True + + +def test_fake_stage_relative_move_within_bounds_does_not_flag(): + stage = FakeStage(limits={"x": (0.0, 100.0)}, positions={"x": 10.0}) + hit = stage.move_relative("x", 5.0) + assert stage.get_position("x") == 15.0 + assert hit is False + + +def test_fake_stage_satisfies_stage_interface(): + assert issubclass(FakeStage, Stage) + assert issubclass(NullStage, Stage) From 66bea7a6dd6034426723859496875f68d02b55e5 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 09:56:48 -0400 Subject: [PATCH 06/13] feat: add AcquisitionEngine with shared monotonic frame index Implements the core acquisition loop (ADR 0001): reads both cameras per tick, assigns a single contiguous frame_index, publishes to FrameHub for display/tracking, and forwards AcquiredPairs to an optional sink. A camera failure on one side yields None for that slot without breaking the index sequence, enabling downstream drop detection via index gaps. Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/core/acquisition.py | 61 +++++++++++++ .../backend/code/tests/test_acquisition.py | 87 +++++++++++++++++++ 2 files changed, 148 insertions(+) create mode 100644 WormSpy/backend/code/core/acquisition.py create mode 100644 WormSpy/backend/code/tests/test_acquisition.py diff --git a/WormSpy/backend/code/core/acquisition.py b/WormSpy/backend/code/core/acquisition.py new file mode 100644 index 0000000..99033e4 --- /dev/null +++ b/WormSpy/backend/code/core/acquisition.py @@ -0,0 +1,61 @@ +import threading +import time +from typing import Callable, Optional + +from core.cameras import Camera +from core.frames import AcquiredPair, FrameHub + +Sink = Callable[[AcquiredPair], None] + + +class AcquisitionEngine: + """Single loop that owns both cameras. Grabs both per tick, assigns a shared monotonic + frame_index, publishes the latest frame of each to the hub for display/tracking, and + forwards the paired result to an optional sink (the recorder). This is the only place that + reads from the cameras, so the two streams cannot drift relative to each other.""" + + LEFT = "left" + RIGHT = "right" + + def __init__(self, left: Camera, right: Camera, hub: FrameHub, + sink: Optional[Sink] = None) -> None: + self._left = left + self._right = right + self._hub = hub + self._sink = sink + self._next_index = 0 + self._stop = threading.Event() + self.ticks_completed = 0 + + def run_once(self) -> AcquiredPair: + index = self._next_index + self._next_index += 1 + ts = time.monotonic() + + left_frame = self._left.read() + right_frame = self._right.read() + + if left_frame is not None: + self._hub.publish(self.LEFT, left_frame) + if right_frame is not None: + self._hub.publish(self.RIGHT, right_frame) + + pair = AcquiredPair(frame_index=index, capture_ts=ts, + left=left_frame, right=right_frame) + if self._sink is not None: + self._sink(pair) + self.ticks_completed += 1 + return pair + + def run(self, max_ticks: Optional[int] = None) -> None: + produced = 0 + while not self._stop.is_set(): + if max_ticks is not None and produced >= max_ticks: + break + if not (self._left.is_open() and self._right.is_open()): + break + self.run_once() + produced += 1 + + def stop(self) -> None: + self._stop.set() diff --git a/WormSpy/backend/code/tests/test_acquisition.py b/WormSpy/backend/code/tests/test_acquisition.py new file mode 100644 index 0000000..146dc6d --- /dev/null +++ b/WormSpy/backend/code/tests/test_acquisition.py @@ -0,0 +1,87 @@ +import numpy as np +from core.cameras import FakeCamera +from core.frames import FrameHub +from core.acquisition import AcquisitionEngine + + +def _imgs(n, base=0): + return [np.full((2, 2), (base + i) % 256, np.uint8) for i in range(n)] + + +def test_run_once_assigns_contiguous_monotonic_indices(): + left = FakeCamera(_imgs(3, base=0)) + right = FakeCamera(_imgs(3, base=100)) + engine = AcquisitionEngine(left, right, FrameHub()) + indices = [] + for _ in range(3): + pair = engine.run_once() + indices.append(pair.frame_index) + assert indices == [0, 1, 2] + + +def test_run_once_pairs_left_and_right_per_tick(): + left = FakeCamera(_imgs(2, base=0)) + right = FakeCamera(_imgs(2, base=100)) + engine = AcquisitionEngine(left, right, FrameHub()) + pair = engine.run_once() + assert int(pair.left.image[0, 0]) == 0 + assert int(pair.right.image[0, 0]) == 100 + + +def test_run_once_publishes_latest_to_hub(): + hub = FrameHub() + left = FakeCamera(_imgs(1, base=5)) + right = FakeCamera(_imgs(1, base=50)) + engine = AcquisitionEngine(left, right, hub) + engine.run_once() + assert int(hub.latest("left").image[0, 0]) == 5 + assert int(hub.latest("right").image[0, 0]) == 50 + + +def test_run_once_feeds_the_sink(): + received = [] + left = FakeCamera(_imgs(2)) + right = FakeCamera(_imgs(2, base=100)) + engine = AcquisitionEngine(left, right, FrameHub(), sink=received.append) + engine.run_once() + engine.run_once() + assert [p.frame_index for p in received] == [0, 1] + + +def test_one_camera_failing_still_increments_index_and_keeps_other(): + left = FakeCamera(_imgs(2), fail_on={0}) + right = FakeCamera(_imgs(2, base=100)) + engine = AcquisitionEngine(left, right, FrameHub()) + pair0 = engine.run_once() + assert pair0.frame_index == 0 + assert pair0.left is None + assert pair0.right is not None + pair1 = engine.run_once() + assert pair1.frame_index == 1 + assert pair1.left is not None + + +def test_device_frame_ids_are_preserved_for_drop_detection(): + left = FakeCamera(_imgs(3), frame_ids=[0, 1, 3]) # camera dropped id 2 + right = FakeCamera(_imgs(3, base=100), frame_ids=[0, 1, 2]) + engine = AcquisitionEngine(left, right, FrameHub()) + left_ids = [engine.run_once().left.device_frame_id for _ in range(3)] + assert left_ids == [0, 1, 3] + + +def test_run_loop_stops_and_drains_until_stream_ends(): + left = FakeCamera(_imgs(4)) + right = FakeCamera(_imgs(4, base=100)) + sink = [] + engine = AcquisitionEngine(left, right, FrameHub(), sink=sink.append) + engine.run(max_ticks=4) + assert [p.frame_index for p in sink] == [0, 1, 2, 3] + + +def test_run_loop_exits_when_stop_called(): + left = FakeCamera(_imgs(1000)) + right = FakeCamera(_imgs(1000, base=1)) + engine = AcquisitionEngine(left, right, FrameHub()) + engine.stop() + engine.run(max_ticks=1000) + assert engine.ticks_completed <= 1 From 7d9eb66005b8daf06657915819cd858eea38073d Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 10:01:48 -0400 Subject: [PATCH 07/13] feat: add pure tracking helpers and Tracker thread Implements find_worm_cms (largest-blob centroid via skimage), simple_to_center (pixel offset to mm move), and Tracker (injected compute callback wired to stage.move_relative), keeping the tracking thread fully decoupled from the AcquisitionEngine per ADR 0001. Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/core/tracking.py | 90 +++++++++++++++++++++ WormSpy/backend/code/tests/test_tracking.py | 82 +++++++++++++++++++ 2 files changed, 172 insertions(+) create mode 100644 WormSpy/backend/code/core/tracking.py create mode 100644 WormSpy/backend/code/tests/test_tracking.py diff --git a/WormSpy/backend/code/core/tracking.py b/WormSpy/backend/code/core/tracking.py new file mode 100644 index 0000000..b8e4109 --- /dev/null +++ b/WormSpy/backend/code/core/tracking.py @@ -0,0 +1,90 @@ +"""Pure tracking helpers (moved from app.py) and the Tracker thread. + +The Tracker reads the latest brightfield frame from the hub, computes a target stage move via +an injected `compute` callback, and commands the stage. Compute is injected so the heavy/slow +algorithm (e.g. DeepLabCut) can run here without ever throttling the AcquisitionEngine.""" + +import threading +import time +from typing import Callable, Optional, Tuple + +import numpy as np +from skimage.measure import label, regionprops + +from core.frames import FrameHub +from core.stage import Stage + + +def find_worm_cms(processed_frame: np.ndarray, factor: int, + initial_coords: Tuple[float, float]) -> Tuple[float, float]: + """Centroid (x=col, y=row) of the largest connected region, rescaled by `factor`. + Falls back to initial_coords when no region is found.""" + labeled = label(processed_frame) + regions = regionprops(labeled) + regions_by_area = sorted(regions, key=lambda r: r.area, reverse=True) + if regions_by_area: + coords = regions_by_area[0].centroid # (row, col) + else: + return initial_coords + coords = (round(coords[1]), round(coords[0])) # -> (x=col, y=row) + return (coords[0] * factor, coords[1] * factor) + + +def simple_to_center(centroid_x: float, centroid_y: float, + resolution: Tuple[float, float], + total_mm_x: float, total_mm_y: float, + orient_x: int, orient_y: int) -> Tuple[float, float]: + """Millimetres the stage must move to bring (centroid_x, centroid_y) to frame center.""" + percent_x = float(centroid_x) / float(resolution[0]) + percent_y = float(centroid_y) / float(resolution[1]) + millis_x = percent_x * total_mm_x + millis_y = percent_y * total_mm_y + move_x = orient_x * (millis_x - total_mm_x / 2) + move_y = orient_y * (millis_y - total_mm_y / 2) + return move_x, move_y + + +# Callback types +EnabledFn = Callable[[], bool] +ModeFn = Callable[[], int] +ComputeFn = Callable[[np.ndarray, int], Tuple[float, float]] # (frame, mode) -> (dx, dy) native +OnMoveFn = Callable[[float, float, bool], None] + + +class Tracker: + """Runs in its own thread. Best-effort consumer of the latest brightfield frame.""" + + def __init__(self, hub: FrameHub, stage: Stage, + enabled: EnabledFn, mode: ModeFn, compute: ComputeFn, + on_move: Optional[OnMoveFn] = None, + stream: str = "left", interval_s: float = 0.0) -> None: + self._hub = hub + self._stage = stage + self._enabled = enabled + self._mode = mode + self._compute = compute + self._on_move = on_move + self._stream = stream + self._interval_s = interval_s + self._stop = threading.Event() + + def step(self) -> None: + if not self._enabled(): + return + frame = self._hub.latest(self._stream) + if frame is None: + return + dx, dy = self._compute(frame.image, self._mode()) + hit_x = self._stage.move_relative("x", dx) + hit_y = self._stage.move_relative("y", dy) + if self._on_move is not None: + self._on_move(dx, dy, hit_x or hit_y) + + def run(self) -> None: + while not self._stop.is_set(): + self.step() + if self._interval_s: + time.sleep(self._interval_s) + + def stop(self) -> None: + self._stop.set() diff --git a/WormSpy/backend/code/tests/test_tracking.py b/WormSpy/backend/code/tests/test_tracking.py new file mode 100644 index 0000000..1f9b7d4 --- /dev/null +++ b/WormSpy/backend/code/tests/test_tracking.py @@ -0,0 +1,82 @@ +import numpy as np +from core.frames import FrameHub, Frame +from core.stage import FakeStage +from core.tracking import find_worm_cms, simple_to_center, Tracker + + +def test_find_worm_cms_returns_centroid_of_largest_blob(): + # 40x40 frame, a bright square blob centered near (10, 20) in (row, col) + frame = np.zeros((40, 40), np.uint8) + frame[8:13, 18:23] = 255 # rows 8-12, cols 18-22 -> centroid ~ (10, 20) + x, y = find_worm_cms(frame, factor=1, initial_coords=(20, 20)) + # returned as (x=col, y=row) + assert abs(x - 20) <= 1 + assert abs(y - 10) <= 1 + + +def test_find_worm_cms_applies_downsample_factor(): + frame = np.zeros((20, 20), np.uint8) + frame[9:11, 9:11] = 255 # centroid ~ (9.5, 9.5) + x, y = find_worm_cms(frame, factor=2, initial_coords=(10, 10)) + assert abs(x - 19) <= 2 # ~9.5 * 2 + assert abs(y - 19) <= 2 + + +def test_find_worm_cms_falls_back_to_initial_when_empty(): + frame = np.zeros((10, 10), np.uint8) + assert find_worm_cms(frame, factor=1, initial_coords=(5, 5)) == (5, 5) + + +def test_simple_to_center_centered_worm_needs_no_move(): + mx, my = simple_to_center(960, 600, resolution=(1920, 1200), + total_mm_x=1.92, total_mm_y=1.2, + orient_x=1, orient_y=-1) + assert abs(mx) < 1e-9 + assert abs(my) < 1e-9 + + +def test_simple_to_center_respects_orientation_sign(): + mx, _ = simple_to_center(1920, 600, resolution=(1920, 1200), + total_mm_x=1.92, total_mm_y=1.2, + orient_x=1, orient_y=-1) + assert mx > 0 + mx_inv, _ = simple_to_center(1920, 600, resolution=(1920, 1200), + total_mm_x=1.92, total_mm_y=1.2, + orient_x=-1, orient_y=-1) + assert mx_inv < 0 + + +def test_tracker_step_does_nothing_when_disabled(): + hub = FrameHub() + hub.publish("left", Frame(np.zeros((40, 40), np.uint8), 0, 0.0)) + stage = FakeStage(limits={"x": (0.0, 1e9), "y": (0.0, 1e9)}, + positions={"x": 100.0, "y": 100.0}) + tracker = Tracker(hub, stage, enabled=lambda: False, mode=lambda: 0, + compute=lambda frame, mode: (5, 5)) + tracker.step() + assert stage.get_position("x") == 100.0 # unchanged + + +def test_tracker_step_commands_stage_when_enabled(): + hub = FrameHub() + hub.publish("left", Frame(np.zeros((40, 40), np.uint8), 0, 0.0)) + stage = FakeStage(limits={"x": (-1e9, 1e9), "y": (-1e9, 1e9)}, + positions={"x": 0.0, "y": 0.0}) + moves = [] + tracker = Tracker(hub, stage, enabled=lambda: True, mode=lambda: 0, + compute=lambda frame, mode: (123.0, -45.0), + on_move=lambda dx, dy, hit: moves.append((dx, dy))) + tracker.step() + assert moves == [(123.0, -45.0)] + assert stage.get_position("x") == 123.0 + assert stage.get_position("y") == -45.0 + + +def test_tracker_step_skips_when_no_frame_available(): + hub = FrameHub() # nothing published + stage = FakeStage(limits={"x": (0.0, 10.0), "y": (0.0, 10.0)}, + positions={"x": 1.0, "y": 1.0}) + tracker = Tracker(hub, stage, enabled=lambda: True, mode=lambda: 0, + compute=lambda frame, mode: (5, 5)) + tracker.step() # must not raise + assert stage.get_position("x") == 1.0 From 242008c354d0a3acd9584654098a473d278bb236 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 10:04:10 -0400 Subject: [PATCH 08/13] feat: add ZaberStage adapter with graceful connect-once fallback Implements core/stage_zaber.py with ZaberStage (lazy zaber_motion imports, opens serial connections once at construction) and connect_stage factory that degrades to NullStage on any connection failure so the server boots without hardware. Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/core/stage_zaber.py | 72 ++++++++++++++++++++++++ WormSpy/backend/code/tests/test_stage.py | 15 +++++ 2 files changed, 87 insertions(+) create mode 100644 WormSpy/backend/code/core/stage_zaber.py diff --git a/WormSpy/backend/code/core/stage_zaber.py b/WormSpy/backend/code/core/stage_zaber.py new file mode 100644 index 0000000..011fbfc --- /dev/null +++ b/WormSpy/backend/code/core/stage_zaber.py @@ -0,0 +1,72 @@ +"""Zaber hardware adapter. Imports zaber_motion lazily so tests never load the SDK.""" + +from typing import Callable, Optional, Tuple + +from core.stage import Stage, NullStage, clamp_native + + +class ZaberStage(Stage): + """Wraps three Zaber axes (x, y on the XY connection; z on the Z connection). + + Connections are opened once at construction and held for the process lifetime.""" + + def __init__(self, xy_port: str, z_port: str) -> None: + from zaber_motion import Library # lazy import + from zaber_motion.ascii import Connection + Library.enable_device_db_store() + + self._xy_conn = Connection.open_serial_port(xy_port) + self._z_conn = Connection.open_serial_port(z_port) + self._xy_conn.enable_alerts() + self._z_conn.enable_alerts() + + horizontal = self._xy_conn.detect_devices() + vertical = self._z_conn.detect_devices() + self._axes = { + "x": horizontal[0].get_axis(1), + "y": horizontal[1].get_axis(1), + "z": vertical[0].get_axis(1), + } + self._limits = { + name: (float(axis.settings.get("limit.min")), + float(axis.settings.get("limit.max"))) + for name, axis in self._axes.items() + } + + def is_connected(self) -> bool: + return True + + def get_position(self, axis: str) -> float: + from zaber_motion import Units + return float(self._axes[axis].get_position(unit=Units.NATIVE)) + + def limits(self, axis: str) -> Tuple[float, float]: + return self._limits[axis] + + def move_relative(self, axis: str, native: float) -> bool: + lo, hi = self._limits[axis] + current = self.get_position(axis) + target, hit = clamp_native(current + native, lo, hi) + delta = int(target - current) + if delta != 0: + self._axes[axis].generic_command_no_response(f"move rel {delta} 10000 5") + return hit + + def move_absolute(self, axis: str, native: float) -> bool: + from zaber_motion import Units + lo, hi = self._limits[axis] + target, hit = clamp_native(native, lo, hi) + self._axes[axis].move_absolute(target, unit=Units.NATIVE, wait_until_idle=False) + return hit + + +def connect_stage(xy_port: str, z_port: str, + connector: Optional[Callable[..., Stage]] = None) -> Stage: + """Open the stage once at startup. On any failure, return a NullStage so the server still + runs (e.g. for UI/dev work with no hardware). `connector` is injectable for tests.""" + factory = connector if connector is not None else (lambda **kw: ZaberStage(**kw)) + try: + return factory(xy_port=xy_port, z_port=z_port) + except Exception as exc: # noqa: BLE001 - degrade gracefully on any hardware/SDK error + print(f"connect_stage: no stage hardware ({exc!r}); using NullStage.") + return NullStage() diff --git a/WormSpy/backend/code/tests/test_stage.py b/WormSpy/backend/code/tests/test_stage.py index edf575d..6ef0446 100644 --- a/WormSpy/backend/code/tests/test_stage.py +++ b/WormSpy/backend/code/tests/test_stage.py @@ -1,5 +1,6 @@ import pytest from core.stage import Stage, NullStage, FakeStage, clamp_native +from core.stage_zaber import connect_stage def test_clamp_within_bounds_is_unchanged(): @@ -48,3 +49,17 @@ def test_fake_stage_relative_move_within_bounds_does_not_flag(): def test_fake_stage_satisfies_stage_interface(): assert issubclass(FakeStage, Stage) assert issubclass(NullStage, Stage) + + +def test_connect_stage_returns_null_stage_when_connection_fails(): + def boom(*_args, **_kwargs): + raise RuntimeError("no serial port") + stage = connect_stage(xy_port="COM6", z_port="COM3", connector=boom) + assert stage.is_connected() is False # NullStage + + +def test_connect_stage_uses_injected_connector_result(): + sentinel = FakeStage(limits={"x": (0.0, 1.0), "y": (0.0, 1.0), "z": (0.0, 1.0)}, + positions={"x": 0.0, "y": 0.0, "z": 0.0}) + stage = connect_stage(xy_port="COM6", z_port="COM3", connector=lambda **_: sentinel) + assert stage is sentinel From a685b5a3d1d3907ea54f7f9ebd8bbbcc522aecb8 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 10:05:49 -0400 Subject: [PATCH 09/13] feat: add SpinnakerCamera adapter configured as the acquisition clock Implements the Camera ABC for FLIR/Spinnaker hardware via EasyPySpin. EasyPySpin is imported lazily inside __init__ so the module is parseable/importable without the SDK installed. The camera is configured as the acquisition clock: fixed AcquisitionFrameRate and exposure with auto-exposure/auto-gain disabled, matching ADR 0001 decision #2. Co-Authored-By: Claude Opus 4.8 --- .../backend/code/core/cameras_spinnaker.py | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 WormSpy/backend/code/core/cameras_spinnaker.py diff --git a/WormSpy/backend/code/core/cameras_spinnaker.py b/WormSpy/backend/code/core/cameras_spinnaker.py new file mode 100644 index 0000000..c363794 --- /dev/null +++ b/WormSpy/backend/code/core/cameras_spinnaker.py @@ -0,0 +1,49 @@ +"""FLIR/Spinnaker camera adapter via EasyPySpin. Imports the SDK lazily so tests don't load it. + +Sets the camera as the acquisition clock: a fixed AcquisitionFrameRate and a fixed exposure, +with auto-exposure/auto-gain off. EasyPySpin exposes these through cv2-style properties; the +device frame id is read from the grabbed image metadata when available, else -1.""" + +import time +from typing import Optional + +import cv2 +import numpy as np + +from core.cameras import Camera +from core.frames import Frame + + +class SpinnakerCamera(Camera): + def __init__(self, identifier, frame_rate: float, exposure_us: float) -> None: + import EasyPySpin # lazy import + self._cap = EasyPySpin.VideoCapture(identifier) + if not self._cap.isOpened(): + print(f"SpinnakerCamera: could not open camera {identifier!r}") + return + # Camera is the clock: disable auto, pin rate + exposure. + self._cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 0) # manual exposure + self._cap.set(cv2.CAP_PROP_EXPOSURE, exposure_us) + self._cap.set(cv2.CAP_PROP_FPS, frame_rate) + self.width = self._cap.get(cv2.CAP_PROP_FRAME_WIDTH) + self.height = self._cap.get(cv2.CAP_PROP_FRAME_HEIGHT) + + def read(self) -> Optional[Frame]: + ok, image = self._cap.read() + ts = time.monotonic() + if not ok or image is None: + return None + if image.dtype != np.uint8: + # Preserve raw for recording elsewhere; this path is display-oriented in Phase 1. + image = cv2.normalize(image, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U) + if len(image.shape) == 3: + image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) + frame_id = int(self._cap.get(cv2.CAP_PROP_POS_FRAMES)) + return Frame(image=image, device_frame_id=frame_id, capture_ts=ts) + + def is_open(self) -> bool: + return self._cap is not None and self._cap.isOpened() + + def release(self) -> None: + if self._cap is not None: + self._cap.release() From b7bba45cc0cee599d9a21f09f51b7114fe3dcff0 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 10:19:59 -0400 Subject: [PATCH 10/13] refactor: own acquisition at startup; serve display from FrameHub (ADR 0001) Co-Authored-By: Claude Opus 4.8 --- WormSpy/backend/code/app.py | 259 ++++++------------------------------ 1 file changed, 43 insertions(+), 216 deletions(-) diff --git a/WormSpy/backend/code/app.py b/WormSpy/backend/code/app.py index 9c38351..7a8e2a4 100644 --- a/WormSpy/backend/code/app.py +++ b/WormSpy/backend/code/app.py @@ -54,6 +54,26 @@ "use_avi_r": True, # Set to True if you want to record the right camera as a compressed avi file, False if you want to record as uncompressed tiff files } +from core.frames import FrameHub +from core.cameras_spinnaker import SpinnakerCamera +from core.stage_zaber import connect_stage +from core.acquisition import AcquisitionEngine +import threading as _threading + +# Shared singletons constructed once at startup (replaces per-request setup). +HUB = FrameHub() +STAGE = connect_stage(XYmotorport, Zmotorport) # NullStage if no hardware +ENGINE = None # set when the live feed starts (cameras chosen in the UI) +ACQ_THREAD = None + +def _start_engine(left_id, right_id): + global ENGINE, ACQ_THREAD + left = SpinnakerCamera(left_id, frame_rate=FPS, exposure_us=10000) + right = SpinnakerCamera(right_id, frame_rate=FPS, exposure_us=10000) + ENGINE = AcquisitionEngine(left, right, HUB, sink=lambda pair: None) # recorder: Phase 2 + ACQ_THREAD = _threading.Thread(target=ENGINE.run, daemon=True) + ACQ_THREAD.start() + # DLC Live Settings: # from dlclive import DLCLive, Processor # dlc_proc = Processor() # Import the DLC NN model @@ -100,217 +120,29 @@ def serve_angular(path): return send_from_directory(app.static_folder, path) return render_template('index.html') +def _mjpeg_from_hub(stream): + def gen(): + while True: + frame = HUB.latest(stream) + if frame is None: + time.sleep(0.03) + continue + ok, buf = cv2.imencode('.jpg', frame.image) # jpeg matches the declared mimetype + if ok: + yield (b'--frame\r\n' + b'Content-Type: image/jpeg\r\n\r\n' + buf.tobytes() + b'\r\n') + time.sleep(0.03) + return Response(gen(), mimetype='multipart/x-mixed-replace; boundary=frame') + @cross_origin() @app.route('/video_feed') def video_feed(): - global leftCam - # Open the video capture - cap = EasyPySpin.VideoCapture(leftCam) # change to BaslerCamera(leftCam) for pypylon - resolution_l = cap.get(cv2.CAP_PROP_FRAME_WIDTH), cap.get(cv2.CAP_PROP_FRAME_HEIGHT) - initial_coords = [(resolution_l[0]/2), resolution_l[1]/2] # INITIALIZE IN CENTER OF FRAME - # Check if the camera is opened successfully - if not cap.isOpened(): - print("Camera can't open\nexit") - return -1 - if stop_stream: - cap.release() - # function to generate a stream of image frames for the tracking video feed - def gen(): - global xMotor, yMotor, zMotor, start_recording, stop_recording, settings, is_tracking, serialPort, start_af, stop_stream, nodeIndex, track_algorithm, is_recording, MAXIMUM_DEVICE_XY_POSITION, MAXIMUM_DEVICE_Z_POSITION - start_af = False - start_recording = False - stop_recording = False - is_recording = False - factor = 2 # Downsample factor change as desired - xPos = 0 - yPos = 0 - with Connection.open_serial_port(XYmotorport) as connection: - with Connection.open_serial_port(Zmotorport) as connection2: - connection.enable_alerts() - connection2.enable_alerts() - horizontal_motors = connection.detect_devices() - vertical_motor = connection2.detect_devices() - print("Found devices on " + XYmotorport.format(len(horizontal_motors))) - print("Found devices on " + Zmotorport.format(len(vertical_motor))) - device_X = horizontal_motors[0] - device_Y = horizontal_motors[1] - device_Z = vertical_motor[0] - xMotor = device_X.get_axis(1) - yMotor = device_Y.get_axis(1) - zMotor = device_Z.get_axis(1) - MAXIMUM_DEVICE_XY_POSITION = device_X.settings.get("limit.max") - MAXIMUM_DEVICE_Z_POSITION = device_Z.settings.get("limit.max") - firstIt = True - while (cap.isOpened()): - success, frame = cap.read() # Read a frame from the video capture - calculated_worm_coords = initial_coords - if success: - if frame.dtype != np.uint8: #check if frame is 8 bit and grayscale and convert if not - display_frame_l = cv2.normalize(frame, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U) - else: - display_frame_l = frame - if len(display_frame_l.shape) == 3: - display_frame_l = cv2.cvtColor(display_frame_l, cv2.COLOR_BGR2GRAY) - frame_downsample = cv2.resize(display_frame_l, (display_frame_l.shape[1] // factor, display_frame_l.shape[0] // factor), interpolation=cv2.INTER_AREA) - xPos = xMotor.get_position(unit=Units.LENGTH_MICROMETRES) - yPos = yMotor.get_position(unit=Units.LENGTH_MICROMETRES) - if is_tracking: ### BUTTON HAS BEEN PRESSED - if track_algorithm == 0: ### BRIGHT BACKGROUND THRESHOLDING - processed_frame = Thresh_Light_Background(frame_downsample) - worm_coords = find_worm_cms(processed_frame, factor, initial_coords) - elif track_algorithm == 1: ### FLUORESCENT THRESHOLDING - processed_frame = Thresh_Fluorescent_Marker(frame_downsample) - worm_coords = find_worm_cms(processed_frame, factor, initial_coords) - elif track_algorithm == 2: ### DEEP LAB CUT TRACKING - height, width = frame_downsample.shape - downsample_size = (int(height), int(width)) - poseArr = DLC_tracking(dlc_live,firstIt,frame_downsample,downsample_size, factor) - posArr = poseArr[:, [0, 1]] - worm_coords = (posArr[nodeIndex, 0] , posArr[nodeIndex, 1]) - calculated_worm_coords = (worm_coords[0] , worm_coords[1]) - # MUTEX POSITION 1 - if not mutex.locked(): - # returns microsteps - trackWorm( - (calculated_worm_coords[0], calculated_worm_coords[1]), xMotor, yMotor, xPos, yPos, resolution_l) # TRACKING FUNCTION - # Reinitialize file recording - if start_recording: - print("Start Left Recording") - start_recording = False - is_recording = True - dt = datetime.now(tz=timeZone) - dtstr = dt.strftime("%d-%m-%Y_%H-%M") - folder_name = settings["filename"] + '_' + dtstr - if settings["use_avi_l"]: - # initialize video writer - fourcc_l = cv2.VideoWriter_fourcc(*'XVID') - parent_path: pathlib.Path = filepathToDirectory(settings["filepath"]) / folder_name - if not parent_path.exists(): - parent_path.mkdir(parents=True, exist_ok=False) - video_writer_l = cv2.VideoWriter(str(parent_path / (folder_name + "_L.avi")), fourcc_l, FPS, (display_frame_l.shape[1], display_frame_l.shape[0]), isColor=False) - else: - tiff_folder: pathlib.Path = filepathToDirectory(settings["filepath"]) / folder_name / 'leftcam_tiffs' - if not tiff_folder.exists(): - tiff_folder.mkdir(parents=True, exist_ok=False) - # Start the writer thread - writer_thread_l = Thread(target=tiff_writer, args=(tiff_folder, frame_queue_left)) - writer_thread_l.start() - dtype = [('timestamp', 'U26'), ('X_position', 'f8'), ('Y_position', 'f8')] - csvDump = np.zeros(0, dtype=dtype) - if is_recording: - time = datetime.now(tz=timeZone) - dt = time.strftime("%H-%M-%S.%f") - csvDump = np.append(csvDump, np.array([(dt, xPos, yPos)], dtype=dtype)) - if settings["use_avi_l"]: - video_writer_l.write(display_frame_l) - else: - frame_queue_left.put((dt, frame)) # Add frame to queue instead of writing directly - if stop_recording: - print("Stopped Left Recording") - if settings["use_avi_l"]: - video_writer_l.release() - else: - frame_queue_left.put((None, None)) - writer_thread_l.join() - csv_file = settings["filename"] + ".csv" - csv_file_path = pathlib.Path(settings["filepath"]) / folder_name / csv_file - header = "timestamp,X_position,Y_position" # Add header - np.savetxt(str(csv_file_path), csvDump, delimiter=",", header=header, comments="", fmt='%s,%f,%f') - plot_worm_path(csv_file_path) # creates a plot of the worm path in the same directory as the csv file - is_recording = False - stop_recording = False - # Change color to rgb from gray to allow for the coloring of circles - display_frame_l = cv2.cvtColor(display_frame_l, cv2.COLOR_GRAY2RGB) - # draw CMS position on frame as green circle - cv2.circle(display_frame_l, (int(calculated_worm_coords[0]), int(calculated_worm_coords[1])), 9, (0, 255, 0), -1) - # add skeleton overlay to image for DLC - #frame = draw_skeleton(frame, posArr) - ret, jpeg = cv2.imencode('.png', display_frame_l) - # Yield the encoded frame - yield (b'--frame\r\n' - b'Content-Type: image/jpeg\r\n\r\n' + jpeg.tobytes() + b'\r\n') - else: - # If the frame was not successfully read, yield a "blank frame - yield (b'--frame\r\n' - b'Content-Type: image/jpeg\r\n\r\n\r\n') - # Return the video feed as a multipart/x-mixed-replace response - return Response(gen(), mimetype='multipart/x-mixed-replace; boundary=frame') + return _mjpeg_from_hub("left") @cross_origin() @app.route('/video_feed_fluorescent') def video_feed_fluorescent(): - global rightCam, stop_stream - cap2 = EasyPySpin.VideoCapture(rightCam) # change to BaslerCamera(rightCam) for pypylon - # Check if the camera is opened successfully - if not cap2.isOpened(): - print("Camera can't open\nexit") - return -1 - if stop_stream: - cap2.release() - def gen(): - global heatmap_enabled, start_recording_r, stop_recording_r, settings, is_tracking, serialPort, hist_frame, hist_max, latest_frame - is_recording = False - while (cap2.isOpened()): - success, frame = cap2.read() - if success: - if frame.dtype != np.uint8: #check if frame is 8 bit and grayscale and convert if not - display_frame_r = cv2.normalize(frame, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U) - else: - display_frame_r = frame - if len(display_frame_r.shape) == 3: - record_frame_r = cv2.cvtColor(display_frame_r, cv2.COLOR_BGR2GRAY) - else: - record_frame_r = display_frame_r - if heatmap_enabled == True: # Apply the jet color map to the frame - display_frame_r = cv2.applyColorMap(display_frame_r, cv2.COLORMAP_JET) # Apply the jet color map to the frame - hist_frame = copy.copy(display_frame_r) #histogram frame - hist_max = np.max(frame) # max value of the frame - latest_frame = display_frame_r.copy() - if start_recording_r: - print("Start Right Recording") - start_recording_r = False - is_recording = True - dt = datetime.now(tz=timeZone) - dtstr = dt.strftime("%d-%m-%Y_%H-%M") - folder_name = settings["filename"] + '_' + dtstr - if settings["use_avi_r"]: - fourcc_r = cv2.VideoWriter_fourcc(*'XVID') - parent_path: pathlib.Path = filepathToDirectory(settings["filepath"]) / folder_name - if not parent_path.exists(): - parent_path.mkdir(parents=True, exist_ok=False) - video_writer_r = cv2.VideoWriter(str(parent_path / (folder_name + "_R.avi")), fourcc_r, FPS, (record_frame_r.shape[1], record_frame_r.shape[0]), isColor=False) - else: - tiff_folder: pathlib.Path = filepathToDirectory(settings["filepath"]) / folder_name / 'rightcam_tiffs' - if not tiff_folder.exists(): - tiff_folder.mkdir(parents=True, exist_ok=False) - # Start the writer thread - writer_thread_r = Thread(target=tiff_writer, args=(tiff_folder, frame_queue_right)) - writer_thread_r.start() - if is_recording: - time = datetime.now(tz=timeZone) - dt = time.strftime("%H-%M-%S.%f") - if settings["use_avi_r"]: - video_writer_r.write(record_frame_r) - else: - frame_queue_right.put((dt, frame)) # Add frame to queue instead of writing directly - if stop_recording_r: - print("Stopped Right Recording") - is_recording = False - stop_recording_r = False - if settings["use_avi_r"]: - video_writer_r.release() - else: - frame_queue_right.put((None, None)) # Signal the writer thread to stop - writer_thread_r.join() # Wait for the writer thread to finish - ret, jpeg = cv2.imencode('.png', display_frame_r) - yield (b'--frame\r\n' - b'Content-Type: image/jpeg\r\n\r\n' + jpeg.tobytes() + b'\r\n') - else: - # If the frame was not successfully read, yield a blank frame - yield (b'--frame\r\n' - b'Content-Type: image/jpeg\r\n\r\n\r\n') - # Return the video feed as a multipart/x-mixed-replace response - return Response(gen(), mimetype='multipart/x-mixed-replace; boundary=frame') + return _mjpeg_from_hub("right") @cross_origin() @app.route("/get_hist") @@ -401,14 +233,12 @@ def stop_live_stream(): return jsonify({"message": "Streams stopped"}) @cross_origin() -@app.route("/camera_settings", methods=['POST']) +@app.route("/camera_settings", methods=['POST']) def camera_settings(): - global leftCam, rightCam, serialPort - # Set the camera settings before starting the video feeds leftCam = request.json['leftCam'] rightCam = request.json['rightCam'] - serialPort = request.json['serialInput'] - return jsonify({"message": "Recording stopped"}) + _start_engine(leftCam, rightCam) + return jsonify({"message": "Engine started"}) @cross_origin() @app.route("/node_index", methods=['POST']) @@ -453,13 +283,10 @@ def toggle_manual(): @cross_origin() @app.route("/move_to_center", methods=['POST']) def move_to_center(): - global xMotor, yMotor - xMax = xMotor.settings.get("limit.max") - yMax = yMotor.settings.get("limit.max") - midX = xMax / 2 - midY = yMax / 2 - xMotor.move_absolute(midX, unit=Units.NATIVE, wait_until_idle=False) - yMotor.move_absolute(midY, unit=Units.NATIVE, wait_until_idle=False) + lo_x, hi_x = STAGE.limits("x") + lo_y, hi_y = STAGE.limits("y") + STAGE.move_absolute("x", (lo_x + hi_x) / 2) + STAGE.move_absolute("y", (lo_y + hi_y) / 2) return jsonify({"message": "Moved to center"}) def calculate_focus_metric(frame): From de1be2084b526ef5cf5619f47d8b2aadb580e71e Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 10:31:42 -0400 Subject: [PATCH 11/13] docs: Phase 2 recording-integrity plan (ADR 0003) Co-Authored-By: Claude Opus 4.8 --- .../2026-06-21-recording-integrity-phase2.md | 930 ++++++++++++++++++ 1 file changed, 930 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-21-recording-integrity-phase2.md diff --git a/docs/superpowers/plans/2026-06-21-recording-integrity-phase2.md b/docs/superpowers/plans/2026-06-21-recording-integrity-phase2.md new file mode 100644 index 0000000..b762438 --- /dev/null +++ b/docs/superpowers/plans/2026-06-21-recording-integrity-phase2.md @@ -0,0 +1,930 @@ +# Recording Integrity (ADR 0003) Implementation Plan — Phase 2 + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Restore recording on the new acquisition architecture as a *frame-accurate, self-describing session*: a CSV spine keyed by `frame_index` (with capture timestamp, both camera frame IDs, stage x/y/z, and tracking state), brightfield written with an intra-frame codec, fluorescence written as 16-bit TIFF, and all disk I/O behind bounded queues so it never silently corrupts timing. + +**Architecture:** Builds on Phase 1 (ADR 0001). The `AcquisitionEngine.sink` is set to `Recorder.consume`, which runs on the acquisition thread but does only cheap work — snapshot a thread-safe telemetry cache, format one CSV row, and hand frames to bounded `QueueWriter` threads. A `StatePoller` thread keeps the telemetry cache (stage x/y/z, tracking mode/flag, last limit-hit) fresh without ever blocking the acquisition loop on a serial read. Frame/CSV writers sit behind small interfaces so the orchestration is unit-tested with fakes; the real cv2/TIFF writers are verified on the rig. + +**Tech Stack:** Python 3.8 (rig) / 3.12 (dev), NumPy, OpenCV (`cv2.VideoWriter`, `cv2.imwrite`), pytest. As in Phase 1, hardware/`cv2` imports live only inside adapter modules/lazy imports so the pure tests run without them. + +**Scope:** Implements ADR 0003 and the "surface the limit-hit" half of Q8 (the clamp itself shipped in Phase 1's `clamp_native`). Does NOT implement Hold Focus (Phase 3 / ADR 0002) or the `/status` endpoint + UI (Phase 4 / Q6) — though the telemetry cache built here is the data source Phase 4 will expose. After this phase, recording works end-to-end again; autofocus and manual mode remain on their Phase-1-deferred state until Phase 3. + +**Depends on:** branch `refactor/acquisition-core-adr0001` (Phase 1) — build Phase 2 on top of it. Phase 1's hardware adapters should ideally be rig-verified first; the pure tasks here do not require it. + +--- + +## File Structure + +All paths relative to `WormSpy/backend/code/`. + +- Create: `core/telemetry.py` — `TelemetrySnapshot` dataclass; thread-safe `Telemetry` cache; `StatePoller` thread. +- Create: `core/recording.py` — pure CSV helpers (`CSV_HEADER`, `csv_row`); `QueueWriter` (bounded queue + worker thread); `FrameWriter` interface + `FakeFrameWriter`; `Recorder` orchestrator. +- Create: `core/recording_writers.py` — real writers: `Cv2VideoWriter` (intra-frame codec), `TiffSequenceWriter` (16-bit), lazy `cv2` import. +- Create: `tests/test_telemetry.py`, `tests/test_recording.py`. +- Modify: `app.py` — build `Telemetry` + `StatePoller` at startup, set `ENGINE` sink to `Recorder.consume`, rewrite `start_recording`/`stop_recording` routes to drive the single `Recorder`, feed tracking state + limit-hit into telemetry. + +--- + +### Task 1: Telemetry cache and snapshot + +**Files:** +- Create: `WormSpy/backend/code/core/telemetry.py` +- Test: `WormSpy/backend/code/tests/test_telemetry.py` + +> Run tests in this sandbox with: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_telemetry.py -v` from `WormSpy/backend/code`. + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_telemetry.py`: + +```python +import threading +from core.telemetry import Telemetry, TelemetrySnapshot + + +def test_default_snapshot_is_zeroed_and_not_tracking(): + snap = Telemetry().snapshot() + assert snap.x == 0.0 and snap.y == 0.0 and snap.z == 0.0 + assert snap.tracking_mode == 0 + assert snap.is_tracking is False + assert snap.limit_hit is False + + +def test_set_position_then_snapshot_reflects_it(): + t = Telemetry() + t.set_position(10.0, 20.0, 30.0) + snap = t.snapshot() + assert (snap.x, snap.y, snap.z) == (10.0, 20.0, 30.0) + + +def test_set_tracking_state_is_reflected(): + t = Telemetry() + t.set_tracking(mode=2, is_tracking=True) + snap = t.snapshot() + assert snap.tracking_mode == 2 + assert snap.is_tracking is True + + +def test_limit_hit_is_recorded(): + t = Telemetry() + t.set_limit_hit(True) + assert t.snapshot().limit_hit is True + + +def test_snapshot_is_immutable_copy(): + t = Telemetry() + t.set_position(1.0, 2.0, 3.0) + snap = t.snapshot() + t.set_position(9.0, 9.0, 9.0) + # the earlier snapshot must not change + assert (snap.x, snap.y, snap.z) == (1.0, 2.0, 3.0) + + +def test_concurrent_updates_do_not_raise(): + t = Telemetry() + def writer(base): + for i in range(1000): + t.set_position(base + i, base + i, base + i) + threads = [threading.Thread(target=writer, args=(b,)) for b in (0, 10000)] + for th in threads: + th.start() + for th in threads: + th.join() + assert t.snapshot() is not None +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_telemetry.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.telemetry'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/telemetry.py`: + +```python +import threading +from dataclasses import dataclass + + +@dataclass(frozen=True) +class TelemetrySnapshot: + """An immutable, point-in-time view of stage + tracking state for one CSV row.""" + x: float + y: float + z: float + tracking_mode: int + is_tracking: bool + limit_hit: bool + + +class Telemetry: + """Thread-safe mutable cache of the latest stage position and tracking state. + + Written by the StatePoller (position) and by control routes (tracking/limit). Read + (non-blocking) by the Recorder at acquisition-tick time and, later, by the /status endpoint.""" + + def __init__(self) -> None: + self._lock = threading.Lock() + self._x = 0.0 + self._y = 0.0 + self._z = 0.0 + self._tracking_mode = 0 + self._is_tracking = False + self._limit_hit = False + + def set_position(self, x: float, y: float, z: float) -> None: + with self._lock: + self._x, self._y, self._z = x, y, z + + def set_tracking(self, mode: int, is_tracking: bool) -> None: + with self._lock: + self._tracking_mode = mode + self._is_tracking = is_tracking + + def set_limit_hit(self, hit: bool) -> None: + with self._lock: + self._limit_hit = hit + + def snapshot(self) -> TelemetrySnapshot: + with self._lock: + return TelemetrySnapshot( + x=self._x, y=self._y, z=self._z, + tracking_mode=self._tracking_mode, + is_tracking=self._is_tracking, + limit_hit=self._limit_hit, + ) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_telemetry.py` +Expected: PASS (6 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/telemetry.py WormSpy/backend/code/tests/test_telemetry.py +git commit -m "feat: add thread-safe Telemetry cache and snapshot" +``` +End commit message with: `Co-Authored-By: Claude Opus 4.8 ` + +--- + +### Task 2: StatePoller thread + +**Files:** +- Modify: `WormSpy/backend/code/core/telemetry.py` +- Test: `WormSpy/backend/code/tests/test_telemetry.py` (extend) +- Reference: Phase 1 `core/stage.py` (`Stage`, `FakeStage`) + +> The poller reads stage position into the telemetry cache at a fixed rate, OFF the acquisition thread, so the acquisition loop never blocks on a serial read. + +- [ ] **Step 1: Write the failing test** + +Append to `tests/test_telemetry.py`: + +```python +from core.stage import FakeStage +from core.telemetry import StatePoller + + +def test_poller_once_copies_stage_position_into_telemetry(): + t = Telemetry() + stage = FakeStage(limits={"x": (0, 1e9), "y": (0, 1e9), "z": (0, 1e9)}, + positions={"x": 5.0, "y": 6.0, "z": 7.0}) + poller = StatePoller(stage, t) + poller.poll_once() + snap = t.snapshot() + assert (snap.x, snap.y, snap.z) == (5.0, 6.0, 7.0) + + +def test_poller_once_is_noop_when_stage_disconnected(): + from core.stage import NullStage + t = Telemetry() + t.set_position(1.0, 2.0, 3.0) + poller = StatePoller(NullStage(), t) + poller.poll_once() + # NullStage is not connected -> poller must not overwrite with zeros + snap = t.snapshot() + assert (snap.x, snap.y, snap.z) == (1.0, 2.0, 3.0) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_telemetry.py -k poller` +Expected: FAIL — `ImportError: cannot import name 'StatePoller'` + +- [ ] **Step 3: Write minimal implementation** + +Append to `core/telemetry.py`: + +```python +import time + +from core.stage import Stage + + +class StatePoller: + """Polls the stage position into a Telemetry cache at a fixed rate, in its own thread. + + Keeps position fresh for the recorder/status without ever blocking the acquisition loop.""" + + def __init__(self, stage: Stage, telemetry: Telemetry, interval_s: float = 0.05) -> None: + self._stage = stage + self._telemetry = telemetry + self._interval_s = interval_s + self._stop = threading.Event() + + def poll_once(self) -> None: + if not self._stage.is_connected(): + return + self._telemetry.set_position( + self._stage.get_position("x"), + self._stage.get_position("y"), + self._stage.get_position("z"), + ) + + def run(self) -> None: + while not self._stop.is_set(): + self.poll_once() + time.sleep(self._interval_s) + + def stop(self) -> None: + self._stop.set() +``` + +(Put the `import time` and `from core.stage import Stage` near the top of the file with the other imports rather than mid-file.) + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_telemetry.py` +Expected: PASS (8 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/telemetry.py WormSpy/backend/code/tests/test_telemetry.py +git commit -m "feat: add StatePoller to keep telemetry position fresh off the acquisition thread" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 3: CSV spine helpers + +**Files:** +- Create: `WormSpy/backend/code/core/recording.py` +- Test: `WormSpy/backend/code/tests/test_recording.py` + +> Pure formatting for the per-frame CSV spine (ADR 0003). One row per acquisition tick, keyed by `frame_index`; dropped frames are detectable as `frame_index` gaps and as camera-frame-id gaps. + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_recording.py`: + +```python +from core.recording import CSV_HEADER, csv_row +from core.telemetry import TelemetrySnapshot + + +def _snap(x=1.0, y=2.0, z=3.0, mode=0, tracking=True, limit=False): + return TelemetrySnapshot(x=x, y=y, z=z, tracking_mode=mode, + is_tracking=tracking, limit_hit=limit) + + +def test_header_lists_the_spine_columns_in_order(): + assert CSV_HEADER == ("frame_index,capture_timestamp,left_frame_id,right_frame_id," + "x_pos,y_pos,z_pos,tracking_mode,is_tracking") + + +def test_csv_row_formats_all_fields(): + row = csv_row(frame_index=5, capture_ts=12.5, left_id=5, right_id=5, snap=_snap()) + assert row == "5,12.500000,5,5,1.000000,2.000000,3.000000,0,True" + + +def test_csv_row_uses_minus_one_for_missing_frame_ids(): + row = csv_row(frame_index=7, capture_ts=0.0, left_id=-1, right_id=3, snap=_snap()) + assert row.startswith("7,0.000000,-1,3,") + + +def test_csv_row_reflects_tracking_state_false(): + row = csv_row(frame_index=1, capture_ts=0.0, left_id=0, right_id=0, + snap=_snap(tracking=False, mode=2)) + assert row.endswith(",2,False") +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.recording'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/recording.py`: + +```python +from core.telemetry import TelemetrySnapshot + +CSV_HEADER = ("frame_index,capture_timestamp,left_frame_id,right_frame_id," + "x_pos,y_pos,z_pos,tracking_mode,is_tracking") + + +def csv_row(frame_index: int, capture_ts: float, left_id: int, right_id: int, + snap: TelemetrySnapshot) -> str: + """Format one spine row. Keep field order identical to CSV_HEADER.""" + return (f"{frame_index},{capture_ts:.6f},{left_id},{right_id}," + f"{snap.x:.6f},{snap.y:.6f},{snap.z:.6f}," + f"{snap.tracking_mode},{snap.is_tracking}") +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py` +Expected: PASS (4 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/recording.py WormSpy/backend/code/tests/test_recording.py +git commit -m "feat: add CSV spine header and row formatting" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 4: QueueWriter (bounded, backpressure) + +**Files:** +- Modify: `WormSpy/backend/code/core/recording.py` +- Test: `WormSpy/backend/code/tests/test_recording.py` (extend) + +> All disk I/O goes through this so a slow disk never silently corrupts timing. Two policies: `block` (recording — back-pressures to camera-level drops, which appear as logged frame-id gaps rather than silent loss) and `drop` (non-critical — counts drops). + +- [ ] **Step 1: Write the failing test** + +Append to `tests/test_recording.py`: + +```python +import threading +from core.recording import QueueWriter + + +def test_queue_writer_writes_all_items_in_order_then_closes(): + written = [] + w = QueueWriter(write_fn=written.append, maxsize=10, on_full="block") + w.start() + for i in range(5): + w.put(i) + w.close() + assert written == [0, 1, 2, 3, 4] + + +def test_queue_writer_drop_policy_counts_drops_when_full(): + started = threading.Event() + release = threading.Event() + def slow_write(item): + started.set() + release.wait(timeout=2) + w = QueueWriter(write_fn=slow_write, maxsize=1, on_full="drop") + w.start() + w.put("a") # taken by worker, which then blocks in slow_write + started.wait(timeout=2) + w.put("b") # fills the queue (size 1) + w.put("c") # queue full -> dropped + w.put("d") # dropped + assert w.dropped >= 1 + release.set() + w.close() + + +def test_queue_writer_block_policy_does_not_drop(): + written = [] + w = QueueWriter(write_fn=written.append, maxsize=1, on_full="block") + w.start() + for i in range(50): + w.put(i) # must block rather than drop + w.close() + assert written == list(range(50)) + assert w.dropped == 0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py -k queue_writer` +Expected: FAIL — `ImportError: cannot import name 'QueueWriter'` + +- [ ] **Step 3: Write minimal implementation** + +Append to `core/recording.py`: + +```python +import queue +import threading +from typing import Callable + + +class QueueWriter: + """Bounded queue + single worker thread that calls write_fn(item) off the caller's thread. + + on_full="block": put() blocks until space (use for the recording — back-pressure surfaces + as camera-level frame-id gaps, never silent loss). on_full="drop": put() drops and counts.""" + + _SENTINEL = object() + + def __init__(self, write_fn: Callable[[object], None], maxsize: int = 256, + on_full: str = "block") -> None: + if on_full not in ("block", "drop"): + raise ValueError("on_full must be 'block' or 'drop'") + self._write = write_fn + self._on_full = on_full + self._q: "queue.Queue" = queue.Queue(maxsize) + self._thread = threading.Thread(target=self._run, daemon=True) + self.dropped = 0 + + def start(self) -> None: + self._thread.start() + + def put(self, item: object) -> None: + if self._on_full == "drop": + try: + self._q.put_nowait(item) + except queue.Full: + self.dropped += 1 + else: + self._q.put(item) + + def _run(self) -> None: + while True: + item = self._q.get() + if item is self._SENTINEL: + break + self._write(item) + + def close(self) -> None: + self._q.put(self._SENTINEL) + self._thread.join() +``` + +(Move the `import queue`, `import threading`, and `from typing import Callable` to the top of the file with the existing imports.) + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py` +Expected: PASS (7 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/recording.py WormSpy/backend/code/tests/test_recording.py +git commit -m "feat: add bounded QueueWriter with block/drop backpressure policies" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 5: FrameWriter interface + FakeFrameWriter, and the Recorder orchestrator + +**Files:** +- Modify: `WormSpy/backend/code/core/recording.py` +- Test: `WormSpy/backend/code/tests/test_recording.py` (extend) +- Reference: Phase 1 `core/frames.py` (`AcquiredPair`, `Frame`), `core/telemetry.py` + +> The Recorder is the engine `sink`. It runs on the acquisition thread but does only cheap work: +> snapshot telemetry, format a CSV row, and `put` frames/rows onto QueueWriters. When not recording +> it ignores pairs. One Recorder = one recording state machine for both cameras (fixes the legacy +> two-state-machine bug). + +- [ ] **Step 1: Write the failing test** + +Append to `tests/test_recording.py`: + +```python +import numpy as np +from core.frames import AcquiredPair, Frame +from core.telemetry import Telemetry +from core.recording import Recorder, FrameWriter, FakeFrameWriter + + +def _pair(idx, left_val=1, right_val=2, left_id=None, right_id=None): + left = Frame(np.full((2, 2), left_val, np.uint8), + left_id if left_id is not None else idx, 0.0) + right = Frame(np.full((2, 2), right_val, np.uint8), + right_id if right_id is not None else idx, 0.0) + return AcquiredPair(frame_index=idx, capture_ts=float(idx), left=left, right=right) + + +def _recorder_with_fakes(): + telem = Telemetry() + left_w = FakeFrameWriter() + right_w = FakeFrameWriter() + rows = [] + rec = Recorder(telemetry=telem, + left_writer_factory=lambda cfg: left_w, + right_writer_factory=lambda cfg: right_w, + csv_sink=rows.append) + return rec, telem, left_w, right_w, rows + + +def test_recorder_ignores_pairs_when_not_recording(): + rec, _, left_w, right_w, rows = _recorder_with_fakes() + rec.consume(_pair(0)) + assert left_w.images == [] and right_w.images == [] and rows == [] + + +def test_recorder_writes_header_then_rows_and_frames_when_recording(): + rec, telem, left_w, right_w, rows = _recorder_with_fakes() + telem.set_position(10.0, 20.0, 30.0) + rec.start_recording(config={"name": "proj"}) + rec.consume(_pair(0)) + rec.consume(_pair(1)) + rec.stop_recording() + assert rows[0].startswith("frame_index,") # header first + assert rows[1].startswith("0,0.000000,0,0,10.000000,20.000000,30.000000,") + assert rows[2].startswith("1,1.000000,1,1,") + assert len(left_w.images) == 2 and len(right_w.images) == 2 + assert left_w.closed and right_w.closed # writers closed on stop + + +def test_recorder_snapshots_position_at_consume_time(): + rec, telem, _, _, rows = _recorder_with_fakes() + rec.start_recording(config={"name": "p"}) + telem.set_position(1.0, 1.0, 1.0) + rec.consume(_pair(0)) + telem.set_position(99.0, 99.0, 99.0) + rec.consume(_pair(1)) + rec.stop_recording() + assert ",1.000000,1.000000,1.000000," in rows[1] # row for idx 0 + assert ",99.000000,99.000000,99.000000," in rows[2] # row for idx 1 + + +def test_recorder_handles_missing_side_with_minus_one_id(): + rec, _, left_w, right_w, rows = _recorder_with_fakes() + rec.start_recording(config={"name": "p"}) + pair = _pair(0) + pair = AcquiredPair(frame_index=0, capture_ts=0.0, left=None, right=pair.right) + rec.consume(pair) + rec.stop_recording() + assert rows[1].startswith("0,0.000000,-1,0,") # left missing -> -1 + assert len(left_w.images) == 0 and len(right_w.images) == 1 + + +def test_recorder_double_start_is_ignored(): + rec, _, _, _, rows = _recorder_with_fakes() + rec.start_recording(config={"name": "p"}) + rec.start_recording(config={"name": "p"}) # no second header + rec.consume(_pair(0)) + rec.stop_recording() + assert rows.count(rows[0]) == 1 # exactly one header line + + +def test_fake_frame_writer_satisfies_interface(): + assert issubclass(FakeFrameWriter, FrameWriter) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py -k recorder` +Expected: FAIL — `ImportError: cannot import name 'Recorder'` + +- [ ] **Step 3: Write minimal implementation** + +Append to `core/recording.py`: + +```python +from abc import ABC, abstractmethod +from typing import Callable, List, Optional + +import numpy as np + +from core.frames import AcquiredPair +from core.telemetry import Telemetry + + +class FrameWriter(ABC): + """Sink for one camera's frames. Real impls wrap cv2; FakeFrameWriter is the test double.""" + + @abstractmethod + def write(self, image: np.ndarray) -> None: + ... + + @abstractmethod + def close(self) -> None: + ... + + +class FakeFrameWriter(FrameWriter): + def __init__(self) -> None: + self.images: List[np.ndarray] = [] + self.closed = False + + def write(self, image: np.ndarray) -> None: + self.images.append(image) + + def close(self) -> None: + self.closed = True + + +WriterFactory = Callable[[dict], FrameWriter] + + +class Recorder: + """Engine sink. Cheap, runs on the acquisition thread; ignores pairs unless recording. + + `csv_sink` receives header + row strings (a QueueWriter.put in production, a list in tests).""" + + def __init__(self, telemetry: Telemetry, + left_writer_factory: WriterFactory, + right_writer_factory: WriterFactory, + csv_sink: Callable[[str], None]) -> None: + self._telemetry = telemetry + self._left_factory = left_writer_factory + self._right_factory = right_writer_factory + self._csv_sink = csv_sink + self._recording = False + self._left: Optional[FrameWriter] = None + self._right: Optional[FrameWriter] = None + + def start_recording(self, config: dict) -> None: + if self._recording: + return + self._left = self._left_factory(config) + self._right = self._right_factory(config) + self._csv_sink(CSV_HEADER) + self._recording = True + + def consume(self, pair: AcquiredPair) -> None: + if not self._recording: + return + snap = self._telemetry.snapshot() + left_id = pair.left.device_frame_id if pair.left is not None else -1 + right_id = pair.right.device_frame_id if pair.right is not None else -1 + self._csv_sink(csv_row(pair.frame_index, pair.capture_ts, left_id, right_id, snap)) + if pair.left is not None: + self._left.write(pair.left.image) + if pair.right is not None: + self._right.write(pair.right.image) + + def stop_recording(self) -> None: + if not self._recording: + return + self._recording = False + if self._left is not None: + self._left.close() + if self._right is not None: + self._right.close() + self._left = None + self._right = None + + @property + def is_recording(self) -> bool: + return self._recording +``` + +(Consolidate the new imports with those already at the top of the file.) + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py` +Expected: PASS (14 passed) + +- [ ] **Step 5: Run the whole suite** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q` +Expected: PASS (Phase 1's 38 + Phase 2 new tests) + +- [ ] **Step 6: Commit** + +```bash +git add WormSpy/backend/code/core/recording.py WormSpy/backend/code/tests/test_recording.py +git commit -m "feat: add Recorder orchestrator with single recording state machine" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 6: Real frame writers (cv2 video + 16-bit TIFF) + +**Files:** +- Create: `WormSpy/backend/code/core/recording_writers.py` +- Reference: legacy `app.py` original recording (`cv2.VideoWriter_fourcc(*'XVID')` → replace with intra-frame codec; 16-bit TIFF via `cv2.imwrite`) + +> Hardware/codec-dependent; verified manually on the rig (Step 4). `cv2` is imported lazily so the +> module imports without OpenCV in the sandbox. Brightfield uses an intra-frame codec (MJPG default, +> FFV1 lossless option) — NOT XVID. Fluorescence frames are written as individual 16-bit TIFFs to +> preserve dynamic range. + +- [ ] **Step 1: Write the implementation** + +Create `core/recording_writers.py`: + +```python +"""Real frame writers. cv2 is imported lazily so this module imports without OpenCV present.""" + +import pathlib +from typing import Tuple + +import numpy as np + +from core.recording import FrameWriter + +# Intra-frame codecs only (each frame stands alone -> frame-accurate behaviour scoring). +CODEC_FOURCC = { + "mjpg": "MJPG", # intra-frame, compressed + "ffv1": "FFV1", # intra-frame, lossless +} + + +class Cv2VideoWriter(FrameWriter): + """Writes grayscale frames to a single video file with an intra-frame codec.""" + + def __init__(self, path: str, fps: float, size: Tuple[int, int], codec: str = "mjpg") -> None: + import cv2 # lazy + fourcc = cv2.VideoWriter_fourcc(*CODEC_FOURCC[codec]) + self._writer = cv2.VideoWriter(path, fourcc, fps, size, isColor=False) + + def write(self, image: np.ndarray) -> None: + self._writer.write(image) + + def close(self) -> None: + self._writer.release() + + +class TiffSequenceWriter(FrameWriter): + """Writes each frame as an individual TIFF (16-bit preserved) into a folder.""" + + def __init__(self, folder: str) -> None: + self._folder = pathlib.Path(folder) + self._folder.mkdir(parents=True, exist_ok=True) + self._n = 0 + + def write(self, image: np.ndarray) -> None: + import cv2 # lazy + cv2.imwrite(str(self._folder / f"frame_{self._n:06d}.tiff"), image) + self._n += 1 + + def close(self) -> None: + pass +``` + +- [ ] **Step 2: Parse check (no cv2 needed)** + +Run: `python -c "import ast; ast.parse(open('core/recording_writers.py').read()); print('parse ok')"` +Expected: `parse ok` + +- [ ] **Step 3: Confirm the unit suite is unaffected** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q` +Expected: PASS (no new tests; nothing regressed) + +- [ ] **Step 4: Manual hardware/codec verification (on the rig)** + +After Task 7 wiring, record a short session and confirm: the brightfield video opens and scrubs +frame-by-frame in a standard player; the fluorescence TIFFs are 16-bit; the CSV has one row per +frame with a header. (Pure-Python check of the TIFF bit depth, on any machine with the files: +`python -c "import cv2; print(cv2.imread('frame_000000.tiff', cv2.IMREAD_UNCHANGED).dtype)"` → +expect `uint16` when the camera supplied 16-bit frames.) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/recording_writers.py +git commit -m "feat: add intra-frame video and 16-bit TIFF frame writers" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 7: Wire the Recorder + telemetry into app.py + +**Files:** +- Modify: `WormSpy/backend/code/app.py` + - Startup block (the Phase 1 wiring after `settings = {...}`): add `Telemetry`, `StatePoller`, `Recorder`, and writer factories; set the engine `sink` to `RECORDER.consume`. + - `start_recording` / `stop_recording` routes: drive the single `Recorder`. + - `toggle_tracking`: also push mode/flag into telemetry. + +> Hardware-wired; validated by recording on the rig (Step 4). `cv2` import already exists at the top +> of app.py. This restores recording on the new architecture. Autofocus/manual remain Phase-3 work. + +- [ ] **Step 1: Extend the startup wiring** + +In `app.py`, in the Phase 1 startup block (right after `STAGE = connect_stage(...)`), add: + +```python +from core.telemetry import Telemetry, StatePoller +from core.recording import Recorder, QueueWriter +from core.recording_writers import Cv2VideoWriter, TiffSequenceWriter +import pathlib as _pathlib + +TELEMETRY = Telemetry() +_poller = StatePoller(STAGE, TELEMETRY) +_threading.Thread(target=_poller.run, daemon=True).start() + +# CSV rows go through a blocking QueueWriter so disk lag never corrupts timing. +_csv_writer = {"w": None} + +def _open_csv_writer(config): + folder = _pathlib.Path(config["filepath"]) / config["folder_name"] + folder.mkdir(parents=True, exist_ok=True) + fh = open(folder / (config["name"] + ".csv"), "w") + w = QueueWriter(write_fn=lambda line: fh.write(line + "\n"), maxsize=4096, on_full="block") + w.start() + _csv_writer["w"] = (w, fh) + return w + +def _left_writer_factory(config): + folder = _pathlib.Path(config["filepath"]) / config["folder_name"] + return Cv2VideoWriter(str(folder / (config["folder_name"] + "_L.avi")), + fps=FPS, size=config["left_size"], codec="mjpg") + +def _right_writer_factory(config): + folder = _pathlib.Path(config["filepath"]) / config["folder_name"] + return TiffSequenceWriter(str(folder / "rightcam_tiffs")) + +RECORDER = Recorder(telemetry=TELEMETRY, + left_writer_factory=_left_writer_factory, + right_writer_factory=_right_writer_factory, + csv_sink=lambda line: _csv_writer["w"][0].put(line)) +``` + +And change the engine construction in `_start_engine` so the sink is the recorder: + +```python + ENGINE = AcquisitionEngine(left, right, HUB, sink=RECORDER.consume) +``` + +- [ ] **Step 2: Rewrite the recording routes to drive the single Recorder** + +Replace the existing `start_record` and `stop_record` route functions with: + +```python +@cross_origin() +@app.route("/start_recording", methods=['POST']) +def start_record(): + dt = datetime.now(tz=timeZone).strftime("%d-%m-%Y_%H-%M") + name = request.json.get("filename", settings["filename"]) + filepath = request.json.get("filepath", settings["filepath"]) + folder_name = f"{name}_{dt}" + left = HUB.latest("left") + left_size = (int(left.image.shape[1]), int(left.image.shape[0])) if left else (0, 0) + config = {"name": name, "filepath": filepath, + "folder_name": folder_name, "left_size": left_size} + _open_csv_writer(config) + RECORDER.start_recording(config) + return jsonify({"message": "Recording started", "folder": folder_name}) + + +@cross_origin() +@app.route("/stop_recording", methods=['POST']) +def stop_record(): + RECORDER.stop_recording() + if _csv_writer["w"] is not None: + w, fh = _csv_writer["w"] + w.close() + fh.close() + _csv_writer["w"] = None + return jsonify({"message": "Recording stopped"}) +``` + +- [ ] **Step 3: Push tracking state into telemetry in `toggle_tracking`** + +In the existing `toggle_tracking` route, after setting `is_tracking` and `track_algorithm`, add: + +```python + TELEMETRY.set_tracking(mode=track_algorithm, is_tracking=is_tracking) +``` + +- [ ] **Step 4: Verify + manual rig check** + +Parse: `python -c "import ast; ast.parse(open('WormSpy/backend/code/app.py').read()); print('parse ok')"` → `parse ok` +Suite: `cd WormSpy/backend/code && PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q` → all pass. +On the rig: start live feed, Start Recording, move the stage / toggle tracking, Stop Recording. Confirm a session folder with `.csv` (header + one row per frame, x/y/z populated, tracking_mode/is_tracking correct), an `_L.avi` that scrubs frame-by-frame, and a `rightcam_tiffs/` folder of 16-bit TIFFs. + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/app.py +git commit -m "refactor: drive recording through Recorder + telemetry (ADR 0003)" +``` +End commit message with the Co-Authored-By trailer. + +--- + +## Self-Review + +**Spec coverage (ADR 0003 + Q8 surfacing):** +- *CSV is the per-frame spine keyed by frame_index, with capture ts, both frame IDs, x/y/z, tracking mode/flag* → Tasks 3 (format) + 5 (Recorder emits header+rows) + 7 (real CSV file). ✓ +- *Dropped frames detectable as gaps* → frame_index is contiguous (Phase 1 engine) and camera frame IDs are logged per row; writer back-pressure uses `block` so loss surfaces as camera-level frame-id gaps, not silent writer drops (Task 4). ✓ +- *Brightfield intra-frame codec instead of XVID; lossless option* → Task 6 (`CODEC_FOURCC` mjpg/ffv1, default mjpg). ✓ +- *Fluorescence uncompressed 16-bit TIFF* → Task 6 (`TiffSequenceWriter`, `IMREAD_UNCHANGED` stays 16-bit). ✓ +- *Bounded queues / backpressure* → Task 4 (`QueueWriter`). ✓ +- *Single recording state machine (fix legacy two-machine bug)* → Task 5 (`Recorder.start/stop`, double-start ignored) + Task 7 (one route pair drives it). ✓ +- *Per-frame position without jittering the sacred acquisition rate* → Tasks 1–2 (Telemetry + StatePoller off-thread) + Recorder snapshots the cache at consume time (Task 5). ✓ +- *Q8 limit-hit surfaced* → `Telemetry.set_limit_hit` (Task 1) is the surface; the Tracker's `on_move(hit)` (Phase 1) and joystick path feed it. (Wiring `on_move`→telemetry is a one-line hook added when the Tracker is wired into app.py — note for the implementer if tracking wiring lands here; otherwise it carries to Phase 3.) + +**Placeholder scan:** No "TBD"/"add error handling"/"similar to" placeholders; pure-logic steps carry complete code; IO-adapter and app.py steps carry complete code + explicit manual verification. + +**Type consistency:** `TelemetrySnapshot(x,y,z,tracking_mode,is_tracking,limit_hit)` used identically in Tasks 1/3/5. `csv_row(frame_index, capture_ts, left_id, right_id, snap)` signature matches between Task 3 definition, its tests, and the Recorder call in Task 5. `FrameWriter.write(image)/close()` consistent across Tasks 5/6. `Recorder(telemetry, left_writer_factory, right_writer_factory, csv_sink)` matches between Task 5 definition, its tests, and Task 7 construction. `QueueWriter(write_fn, maxsize, on_full)` consistent across Tasks 4/7. + +**Known follow-ups (not gaps):** wiring the Tracker's `on_move`→`set_limit_hit` and the real per-mode tracking `compute` are completed when tracking is restored (here or Phase 3); the `/status` endpoint that exposes `Telemetry` is Phase 4; FFV1 selection is exposed in the UI in Phase 4 (codec arg already supported). From d5bedf464e1ad12d8ee937724cc9ab1dc3793ad6 Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 10:38:38 -0400 Subject: [PATCH 12/13] docs: Phase 3 (control + Hold Focus) and Phase 4 (status + UI) plans Co-Authored-By: Claude Opus 4.8 --- ...026-06-21-control-and-hold-focus-phase3.md | 652 ++++++++++++++++++ .../2026-06-21-status-and-ui-truth-phase4.md | 541 +++++++++++++++ 2 files changed, 1193 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-21-control-and-hold-focus-phase3.md create mode 100644 docs/superpowers/plans/2026-06-21-status-and-ui-truth-phase4.md diff --git a/docs/superpowers/plans/2026-06-21-control-and-hold-focus-phase3.md b/docs/superpowers/plans/2026-06-21-control-and-hold-focus-phase3.md new file mode 100644 index 0000000..68db5a5 --- /dev/null +++ b/docs/superpowers/plans/2026-06-21-control-and-hold-focus-phase3.md @@ -0,0 +1,652 @@ +# Stage Control Restoration & Hold Focus (ADR 0002) — Phase 3 + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** Restore the stage-control features that Phase 1 deliberately orphaned (auto-tracking, manual joystick) on the new architecture, and replace the broken PID "autofocus" with the honest **Hold Focus** model from ADR 0002 (operator sets focus; software holds/restores it; optional opt-in brightfield drift compensation). + +**Architecture:** Tracking compute algorithms move into a pure `core/tracking_algorithms.py` and are injected into the Phase-1 `Tracker` as its `compute` callback, so the heavy path (DLC) still runs off the acquisition loop. Focus becomes `core/focus.py`: a numpy-only sharpness metric, a `FocusHold` (store/return a z), and an opt-in `DriftCompensator` (brightfield hill-climb that nudges z). The broken `PIDController`/`continuous_autofocus`/`calculate_focus_metric` are deleted. app.py wires tracker + focus + manual joystick to the shared `STAGE`/`HUB`/`TELEMETRY` from Phases 1–2. + +**Tech Stack:** Python 3.8/3.12, NumPy, scikit-image (already used), pygame (joystick), pytest. Focus metric is numpy-only so it is testable without OpenCV. + +**Scope:** Implements ADR 0002 and restores auto-tracking + manual joystick. Closes the Q8 limit-hit loop by feeding the Tracker's `on_move(hit)` into `Telemetry.set_limit_hit`. Does NOT touch the Angular UI or the `/status` endpoint (Phase 4) — though it adds the backend routes the Phase 4 UI will call. + +**Depends on:** Phases 1 (`core/` engine, Tracker, Stage, HUB) and 2 (`Telemetry`). Build on the same branch. + +--- + +## File Structure + +All paths relative to `WormSpy/backend/code/`. Run tests with `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest` from that dir. + +- Create: `core/focus.py` — `focus_metric` (numpy Laplacian variance); `FocusHold`; `DriftCompensator`. +- Create: `core/tracking_algorithms.py` — `thresh_light_background`, `thresh_fluorescent_marker`, `make_compute` (returns a `Tracker` `compute` callback). +- Create: `tests/test_focus.py`, `tests/test_tracking_algorithms.py`. +- Modify: `app.py` — delete dead PID autofocus; construct `Tracker` + `FocusHold` + `DriftCompensator`; restore manual joystick on `STAGE`; routes `/set_focus`, `/return_to_focus`, `/toggle_drift_comp` (replaces `/toggle_af`), and feed tracking limit-hit into telemetry. + +--- + +### Task 1: Focus metric and FocusHold + +**Files:** +- Create: `WormSpy/backend/code/core/focus.py` +- Test: `WormSpy/backend/code/tests/test_focus.py` +- Reference: Phase 1 `core/stage.py` (`FakeStage`) + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_focus.py`: + +```python +import numpy as np +from core.focus import focus_metric, FocusHold +from core.stage import FakeStage, NullStage + + +def test_focus_metric_higher_for_sharper_image(): + blurry = np.full((64, 64), 128, np.uint8) + blurry[20:44, 20:44] = 130 # very low contrast edge + sharp = np.zeros((64, 64), np.uint8) + sharp[20:44, 20:44] = 255 # hard high-contrast edge + assert focus_metric(sharp) > focus_metric(blurry) + + +def test_focus_metric_zero_for_flat_image(): + assert focus_metric(np.full((32, 32), 100, np.uint8)) == 0.0 + + +def _stage(z=50.0): + return FakeStage(limits={"x": (0, 1e9), "y": (0, 1e9), "z": (0.0, 100.0)}, + positions={"x": 0.0, "y": 0.0, "z": z}) + + +def test_focus_hold_has_no_focus_before_set(): + fh = FocusHold(_stage()) + assert fh.has_focus is False + assert fh.stored_z is None + + +def test_set_focus_stores_current_z(): + stage = _stage(z=42.0) + fh = FocusHold(stage) + fh.set_focus() + assert fh.has_focus is True + assert fh.stored_z == 42.0 + + +def test_return_to_focus_moves_stage_back_to_stored_z(): + stage = _stage(z=42.0) + fh = FocusHold(stage) + fh.set_focus() + stage.move_relative("z", 10.0) # drift to 52 + assert stage.get_position("z") == 52.0 + fh.return_to_focus() + assert stage.get_position("z") == 42.0 + + +def test_return_to_focus_is_noop_when_focus_not_set(): + stage = _stage(z=42.0) + fh = FocusHold(stage) + fh.return_to_focus() # must not raise / move + assert stage.get_position("z") == 42.0 + + +def test_set_focus_noop_when_stage_disconnected(): + fh = FocusHold(NullStage()) + fh.set_focus() + assert fh.has_focus is False +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_focus.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.focus'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/focus.py`: + +```python +from typing import Optional + +import numpy as np + +from core.stage import Stage + + +def focus_metric(image: np.ndarray) -> float: + """Sharpness = variance of a discrete Laplacian. Numpy-only (no OpenCV) so it is testable + without hardware libs. Higher = sharper. Flat images return 0.0.""" + img = image.astype(np.float64) + lap = (-4.0 * img + + np.roll(img, 1, axis=0) + np.roll(img, -1, axis=0) + + np.roll(img, 1, axis=1) + np.roll(img, -1, axis=1)) + # trim the 1px border where np.roll wraps, to avoid edge artifacts + inner = lap[1:-1, 1:-1] + return float(inner.var()) + + +class FocusHold: + """Operator-set focus. Stores a z and can return the stage to it. No sensing, no dither.""" + + def __init__(self, stage: Stage) -> None: + self._stage = stage + self._stored_z: Optional[float] = None + self._ref_metric: Optional[float] = None + + @property + def has_focus(self) -> bool: + return self._stored_z is not None + + @property + def stored_z(self) -> Optional[float]: + return self._stored_z + + @property + def ref_metric(self) -> Optional[float]: + return self._ref_metric + + def set_focus(self, ref_metric: Optional[float] = None) -> None: + """Store the current z (and an optional brightfield reference metric for drift comp).""" + if not self._stage.is_connected(): + return + self._stored_z = self._stage.get_position("z") + self._ref_metric = ref_metric + + def return_to_focus(self) -> None: + if self._stored_z is None: + return + self._stage.move_absolute("z", self._stored_z) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_focus.py` +Expected: PASS (7 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/focus.py WormSpy/backend/code/tests/test_focus.py +git commit -m "feat: add numpy focus metric and FocusHold (set/return z)" +``` +End commit message with: `Co-Authored-By: Claude Opus 4.8 ` + +--- + +### Task 2: DriftCompensator (opt-in brightfield hill-climb) + +**Files:** +- Modify: `WormSpy/backend/code/core/focus.py` +- Test: `WormSpy/backend/code/tests/test_focus.py` (extend) + +> The opt-in (B) path from ADR 0002. Hill-climbs the z-stage to keep the brightfield sharpness at +> the reference captured at focus-set time. Dithers the shared stage (documented artifact). The +> metric source is injected as a `sample_metric()` callable so the climb logic is testable without +> images: a unimodal metric-of-z lets us assert convergence. + +- [ ] **Step 1: Write the failing test** + +Append to `tests/test_focus.py`: + +```python +from core.focus import DriftCompensator + + +def test_drift_compensator_climbs_toward_metric_peak(): + # metric peaks at z == 60; FakeStage starts at 50. The compensator should climb toward 60. + stage = FakeStage(limits={"x": (0, 1e9), "y": (0, 1e9), "z": (0.0, 100.0)}, + positions={"x": 0.0, "y": 0.0, "z": 50.0}) + + def sample_metric(): + z = stage.get_position("z") + return 1000.0 - (z - 60.0) ** 2 # unimodal, peak at z=60 + + comp = DriftCompensator(stage, sample_metric, step_native=1.0) + for _ in range(100): + comp.step() + assert abs(stage.get_position("z") - 60.0) <= 2.0 + + +def test_drift_compensator_step_respects_stage_limits(): + stage = FakeStage(limits={"x": (0, 1e9), "y": (0, 1e9), "z": (0.0, 5.0)}, + positions={"x": 0.0, "y": 0.0, "z": 5.0}) + # metric rewards going higher, but z is pinned at its max of 5.0 + comp = DriftCompensator(stage, lambda: stage.get_position("z"), step_native=1.0) + for _ in range(10): + comp.step() + assert stage.get_position("z") <= 5.0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_focus.py -k drift` +Expected: FAIL — `ImportError: cannot import name 'DriftCompensator'` + +- [ ] **Step 3: Write minimal implementation** + +Append to `core/focus.py`: + +```python +from typing import Callable + + +class DriftCompensator: + """Opt-in brightfield hill-climb. Each step() nudges z by ±step_native, keeping the direction + that improved the metric (flips on a decrease). Stateful across calls. Caller is responsible + for logging z per frame and for clearly labelling the focus/intensity artifact this introduces.""" + + def __init__(self, stage, sample_metric: Callable[[], float], step_native: float = 1.0) -> None: + self._stage = stage + self._sample = sample_metric + self._step = step_native + self._direction = 1 + self._last_metric: Optional[float] = None + + def step(self) -> None: + metric = self._sample() + if self._last_metric is not None and metric < self._last_metric: + self._direction = -self._direction # got worse -> reverse + self._stage.move_relative("z", self._direction * self._step) + self._last_metric = metric +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_focus.py` +Expected: PASS (9 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/focus.py WormSpy/backend/code/tests/test_focus.py +git commit -m "feat: add opt-in DriftCompensator brightfield hill-climb" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 3: Port tracking compute algorithms + +**Files:** +- Create: `WormSpy/backend/code/core/tracking_algorithms.py` +- Test: `WormSpy/backend/code/tests/test_tracking_algorithms.py` +- Reference: legacy `app.py` `Thresh_Light_Background`, `Thresh_Fluorescent_Marker`, `trackWorm` (mm→native conversion via `MM_MST`, orientation, `simple_to_center`). + +> These become the `Tracker.compute` callback: `(frame_image, mode) -> (dx, dy)` in native stage +> microsteps. Preprocessing uses scikit-image (`threshold_yen`) and numpy; the centroid + geometry +> reuse Phase 1's `find_worm_cms`/`simple_to_center`. DLC (mode 2) is guarded behind an injected +> pose function so this module never imports DeepLabCut. + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_tracking_algorithms.py`: + +```python +import numpy as np +from core.tracking_algorithms import (thresh_light_background, + thresh_fluorescent_marker, make_compute) + + +def test_thresh_light_background_returns_binary_uint8(): + frame = np.random.randint(0, 255, (64, 64), np.uint8) + out = thresh_light_background(frame) + assert out.dtype == np.uint8 + assert set(np.unique(out)).issubset({0, 255}) + + +def test_thresh_fluorescent_marker_isolates_bright_blob(): + frame = np.zeros((64, 64), np.uint8) + frame[28:36, 28:36] = 255 + out = thresh_fluorescent_marker(frame) + assert out.dtype == np.uint8 + assert out[32, 32] == 255 # bright blob kept + assert out[0, 0] == 0 # dark background dropped + + +def test_make_compute_centered_blob_yields_near_zero_move(): + # bright blob at frame center -> minimal stage move + cfg = dict(factor=1, resolution=(64, 64), total_mm_x=1.92, total_mm_y=1.2, + mm_microsteps=20997, orient_x=1, orient_y=-1, gain=1.0) + compute = make_compute(cfg) + frame = np.zeros((64, 64), np.uint8) + frame[30:34, 30:34] = 255 # ~center + dx, dy = compute(frame, 1) # mode 1 = fluorescent marker + assert abs(dx) < cfg["mm_microsteps"] * 0.1 + assert abs(dy) < cfg["mm_microsteps"] * 0.1 + + +def test_make_compute_offcenter_blob_yields_proportional_move(): + cfg = dict(factor=1, resolution=(64, 64), total_mm_x=1.92, total_mm_y=1.2, + mm_microsteps=20997, orient_x=1, orient_y=-1, gain=1.0) + compute = make_compute(cfg) + frame = np.zeros((64, 64), np.uint8) + frame[30:34, 56:60] = 255 # far right -> positive x move (orient_x=1) + dx, _ = compute(frame, 1) + assert dx > 0 + + +def test_make_compute_dlc_uses_injected_pose_fn(): + cfg = dict(factor=1, resolution=(64, 64), total_mm_x=1.92, total_mm_y=1.2, + mm_microsteps=20997, orient_x=1, orient_y=-1, gain=1.0, + dlc_pose_fn=lambda img: (32.0, 32.0), node_index=0) + compute = make_compute(cfg) + dx, dy = compute(np.zeros((64, 64), np.uint8), 2) # mode 2 = DLC + assert abs(dx) < cfg["mm_microsteps"] * 0.1 # pose at center -> ~no move +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_tracking_algorithms.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.tracking_algorithms'` + +- [ ] **Step 3: Write minimal implementation** + +Create `core/tracking_algorithms.py`: + +```python +"""Per-mode tracking compute, ported from legacy app.py. Produces a Tracker `compute` callback +returning (dx, dy) in native stage microsteps. scikit-image + numpy only; DLC is injected.""" + +import math +from typing import Callable, Tuple + +import numpy as np +from scipy import ndimage as _ndi # for gaussian blur without cv2 +from skimage.filters import threshold_yen + +from core.tracking import find_worm_cms, simple_to_center + + +def thresh_light_background(frame: np.ndarray) -> np.ndarray: + """Bright-background thresholding -> binary (0/255) image of the dark worm body.""" + blurred = _ndi.gaussian_filter(frame, sigma=4) + thr = blurred.mean() - 0.5 * blurred.std() # dark object on bright field + out = (blurred < thr).astype(np.uint8) * 255 + out = _ndi.binary_erosion(out > 0, iterations=3) + out = _ndi.binary_dilation(out, iterations=1) + return out.astype(np.uint8) * 255 + + +def thresh_fluorescent_marker(frame: np.ndarray) -> np.ndarray: + """Yen-threshold a fluorescent marker -> binary (0/255) of the bright blob.""" + gamma = ((frame / 255.0) ** 1.5 * 255).astype(np.uint8) + thr = threshold_yen(gamma) if gamma.max() > gamma.min() else 255 + return (gamma > thr).astype(np.uint8) * 255 + + +def _mm_to_native(centroid_xy: Tuple[float, float], cfg: dict) -> Tuple[float, float]: + mm_x, mm_y = simple_to_center(centroid_xy[0], centroid_xy[1], + resolution=cfg["resolution"], + total_mm_x=cfg["total_mm_x"], total_mm_y=cfg["total_mm_y"], + orient_x=cfg["orient_x"], orient_y=cfg["orient_y"]) + gain = cfg.get("gain", 1.0) + return (mm_x * cfg["mm_microsteps"] * gain, mm_y * cfg["mm_microsteps"] * gain) + + +def make_compute(cfg: dict) -> Callable[[np.ndarray, int], Tuple[float, float]]: + """Build the Tracker compute callback. cfg keys: factor, resolution, total_mm_x/y, + mm_microsteps, orient_x/y, gain; optional dlc_pose_fn + node_index for mode 2.""" + center = (cfg["resolution"][0] / 2.0, cfg["resolution"][1] / 2.0) + + def compute(frame: np.ndarray, mode: int) -> Tuple[float, float]: + if mode == 0: + processed = thresh_light_background(frame) + coords = find_worm_cms(processed, cfg["factor"], center) + elif mode == 1: + processed = thresh_fluorescent_marker(frame) + coords = find_worm_cms(processed, cfg["factor"], center) + elif mode == 2: + pose_fn = cfg.get("dlc_pose_fn") + if pose_fn is None: + return (0.0, 0.0) + coords = pose_fn(frame) # (x, y) in frame pixels + else: + return (0.0, 0.0) + if coords is None or (isinstance(coords[0], float) and math.isnan(coords[0])): + return (0.0, 0.0) + return _mm_to_native(coords, cfg) + + return compute +``` + +NOTE for implementer: legacy used OpenCV (`cv2.GaussianBlur`/`adaptiveThreshold`/`erode`). This port +uses `scipy.ndimage` + a mean/std threshold so it is testable without OpenCV and has no behavioural +dependency on cv2. If the rig shows the bright-background mode under/over-segments, tune the +`mean - 0.5*std` threshold and the erosion/dilation iterations — these are the knobs. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_tracking_algorithms.py` +Expected: PASS (5 passed) + +- [ ] **Step 5: Run full suite** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q` +Expected: PASS + +- [ ] **Step 6: Commit** + +```bash +git add WormSpy/backend/code/core/tracking_algorithms.py WormSpy/backend/code/tests/test_tracking_algorithms.py +git commit -m "feat: port per-mode tracking compute as Tracker compute callback" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 4: Delete dead PID autofocus; add Hold Focus routes + +**Files:** +- Modify: `WormSpy/backend/code/app.py` + +> Removes `PIDController`, `continuous_autofocus`, `calculate_focus_metric`, `start_autofocus` and +> the old `/toggle_af` behaviour. Adds `FocusHold` + `DriftCompensator` at startup and three routes. +> Hardware-adjacent; verified on the rig (Step 4). + +- [ ] **Step 1: Delete the dead autofocus code** + +In `app.py`, delete the `PIDController` class, `calculate_focus_metric`, `continuous_autofocus`, and +`start_autofocus` functions in full. Remove the `af_enabled`/`af_thread`/`latest_frame` globals that +only the PID used. + +- [ ] **Step 2: Construct focus objects at startup** + +In the startup block (after `RECORDER = ...` from Phase 2), add: + +```python +from core.focus import FocusHold, DriftCompensator, focus_metric + +FOCUS = FocusHold(STAGE) +_drift = {"comp": None, "thread": None, "on": False} + +def _drift_sample(): + frame = HUB.latest("left") + return focus_metric(frame.image) if frame is not None else 0.0 +``` + +- [ ] **Step 3: Replace the `/toggle_af` route with Hold Focus routes** + +Remove the old `toggle_af` route. Add: + +```python +@cross_origin() +@app.route("/set_focus", methods=['POST']) +def set_focus(): + FOCUS.set_focus(ref_metric=_drift_sample()) + return jsonify({"message": "Focus set", "z": FOCUS.stored_z}) + + +@cross_origin() +@app.route("/return_to_focus", methods=['POST']) +def return_to_focus(): + FOCUS.return_to_focus() + return jsonify({"message": "Returned to focus", "z": FOCUS.stored_z}) + + +@cross_origin() +@app.route("/toggle_drift_comp", methods=['POST']) +def toggle_drift_comp(): + want = request.json.get('drift_enabled') == "True" + if want and not _drift["on"]: + comp = DriftCompensator(STAGE, _drift_sample, step_native=2.0) + _drift["comp"] = comp + _drift["on"] = True + def _loop(): + import time as _t + while _drift["on"]: + comp.step() + _t.sleep(0.2) + t = _threading.Thread(target=_loop, daemon=True) + _drift["thread"] = t + t.start() + elif not want and _drift["on"]: + _drift["on"] = False + return jsonify({"drift_enabled": _drift["on"]}) +``` + +- [ ] **Step 4: Verify + manual rig check** + +Parse: `python -c "import ast; ast.parse(open('WormSpy/backend/code/app.py').read()); print('parse ok')"` → `parse ok`. +Suite: `cd WormSpy/backend/code && PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q` → all pass. +On the rig: focus the neuron manually with the joystick, click Set Focus; drift the stage, click Return to Focus → stage snaps back to stored z. Enable drift comp and confirm small z dithering that holds focus (and that you would log z per frame via the CSV from Phase 2). + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/app.py +git commit -m "refactor: replace PID autofocus with Hold Focus + drift comp routes (ADR 0002)" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 5: Wire the Tracker (restore auto-tracking) + +**Files:** +- Modify: `WormSpy/backend/code/app.py` + +> Restores auto-tracking on the new architecture and closes the Q8 limit-hit loop. Hardware-verified +> on the rig. + +- [ ] **Step 1: Construct + start the Tracker at startup** + +In the startup block, after `FOCUS = ...`, add: + +```python +from core.tracking import Tracker +from core.tracking_algorithms import make_compute + +_track_cfg = dict(factor=2, resolution=(1920, 1200), + total_mm_x=TOTAL_MM_X, total_mm_y=TOTAL_MM_Y, + mm_microsteps=MM_MST, orient_x=ZABER_ORIENTATION_X, + orient_y=ZABER_ORIENTATION_Y, gain=0.2) # 0.2 ~ legacy /5 damping +TRACKER = Tracker(HUB, STAGE, + enabled=lambda: is_tracking, + mode=lambda: track_algorithm, + compute=make_compute(_track_cfg), + on_move=lambda dx, dy, hit: TELEMETRY.set_limit_hit(hit), + stream="left", interval_s=0.0) +_threading.Thread(target=TRACKER.run, daemon=True).start() +``` + +(Use the real frame resolution if you prefer: read it from `HUB.latest("left")` lazily inside +`compute` config — but the fixed `resolution` here matches the legacy default and is fine for v1.) + +- [ ] **Step 2: Confirm `toggle_tracking` only updates flags** + +Ensure the existing `toggle_tracking` route sets the `is_tracking`/`track_algorithm` globals and +`TELEMETRY.set_tracking(...)` (added in Phase 2) — and nothing else. The Tracker thread reads those +globals via its `enabled`/`mode` callbacks; no per-request tracking work remains. + +- [ ] **Step 3: Verify + manual rig check** + +Parse + suite as before. On the rig: enable tracking (mode 0/1), confirm the stage follows a moving +target and that when the worm reaches a stage limit the limit-hit flag is set (observable later via +Phase 4 `/status`, or temporarily via a log line). + +- [ ] **Step 4: Commit** + +```bash +git add WormSpy/backend/code/app.py +git commit -m "refactor: restore auto-tracking via Tracker thread (ADR 0001/Q8)" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 6: Restore the manual joystick on STAGE + +**Files:** +- Modify: `WormSpy/backend/code/app.py` +- Reference: legacy `start_controller` (pygame axes → motor moves; X/A/B buttons toggled tracking/af/recording). + +> Rewrites the manual joystick loop to drive the shared `STAGE` (the legacy version used the removed +> `xMotor/yMotor/zMotor` globals). Joystick + stage; verified on the rig. + +- [ ] **Step 1: Rewrite `start_controller` to use STAGE** + +Replace the legacy `start_controller` body so axis input maps to `STAGE.move_relative`: + +```python +def start_controller(): + import pygame + global isManualEnabled, is_tracking + pygame.init() + pygame.joystick.init() + if pygame.joystick.get_count() == 0: + print("No joystick connected.") + return + js = pygame.joystick.Joystick(0) + js.init() + last_x = False + while isManualEnabled: + pygame.event.pump() + ix, iy, iz = js.get_axis(0), js.get_axis(1), js.get_axis(3) + if ix: + STAGE.move_relative("x", int(ix * 2000)) + if iy: + STAGE.move_relative("y", int(-iy * 2000)) + if iz: + STAGE.move_relative("z", int(iz * 100)) + if js.get_numbuttons() > 2: + cur_x = js.get_button(2) + if cur_x and not last_x: + is_tracking = not is_tracking + TELEMETRY.set_tracking(mode=track_algorithm, is_tracking=is_tracking) + last_x = cur_x + pygame.quit() +``` + +(Keep the existing `toggle_manual` route, which starts this in a thread when manual mode is enabled.) + +- [ ] **Step 2: Verify + manual rig check** + +Parse + suite. On the rig with a joystick: enable manual mode, confirm the sticks move XY and Z, and +the X button toggles tracking. Confirm manual moves and the tracker do not fight (the Stage clamps +both; if you observe contention, gate the tracker's `enabled` to also require `not isManualEnabled`). + +- [ ] **Step 3: Commit** + +```bash +git add WormSpy/backend/code/app.py +git commit -m "refactor: restore manual joystick control on shared STAGE" +``` +End commit message with the Co-Authored-By trailer. + +--- + +## Self-Review + +**Spec coverage (ADR 0002 + control restoration + Q8):** +- *Remove broken PID autofocus* → Task 4 Step 1. ✓ +- *Hold/Restore focus (default)* → Tasks 1 (`FocusHold`) + 4 (`/set_focus`, `/return_to_focus`). ✓ +- *Opt-in drift compensation, logs z, labelled artifact* → Tasks 2 (`DriftCompensator`) + 4 (`/toggle_drift_comp`); per-frame z logged via Phase 2 CSV. ✓ +- *Restore auto-tracking for all modes off the acquisition loop* → Tasks 3 (compute) + 5 (Tracker wiring). ✓ +- *Q8 limit-hit surfaced* → Task 5 `on_move=lambda ...: TELEMETRY.set_limit_hit(hit)`. ✓ +- *Restore manual joystick* → Task 6. ✓ + +**Placeholder scan:** No "TBD"/"add error handling"/"similar to". Pure-logic tasks carry full code + tests; app.py/joystick tasks carry full code + explicit manual verification (hardware). + +**Type consistency:** `focus_metric(image)->float`, `FocusHold(stage)` with `set_focus/return_to_focus/stored_z/has_focus`, `DriftCompensator(stage, sample_metric, step_native)` consistent across Tasks 1/2/4. `make_compute(cfg)->compute(frame, mode)->(dx,dy)` matches the Phase-1 `Tracker` `ComputeFn` signature and is used in Task 5. `Telemetry.set_limit_hit/set_tracking` match Phase 2 definitions. + +**Known follow-ups (not gaps):** real DLC pose function injection (`dlc_pose_fn`) is left to the user's model (guarded so absence = no DLC move); the bright-background threshold constants may need rig tuning (called out in Task 3); UI controls for the new focus/tracking routes are Phase 4. diff --git a/docs/superpowers/plans/2026-06-21-status-and-ui-truth-phase4.md b/docs/superpowers/plans/2026-06-21-status-and-ui-truth-phase4.md new file mode 100644 index 0000000..fe092f4 --- /dev/null +++ b/docs/superpowers/plans/2026-06-21-status-and-ui-truth-phase4.md @@ -0,0 +1,541 @@ +# Backend Status, UI Truth & Cleanup (Q6) — Phase 4 + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** Make the backend the single source of truth via a polled `/status` endpoint, and make the Angular UI reflect that truth (so a browser refresh no longer lies), surface errors and live telemetry (recording timer, frame count, z, dropped/limit warnings), expose the new Hold Focus controls, and delete the dead code accumulated across the refactor. + +**Architecture:** A pure `core/status.py:build_status(...)` assembles an authoritative state dict from the existing singletons (`ENGINE`, `RECORDER`, `TELEMETRY`, `STAGE`); a thin `/status` route serializes it. The Recorder gains `frames_written` + elapsed counters. In Angular, a `StatusService` polls `/status` at ~2 Hz and the `live-feed` component binds its toggles/readouts to that stream instead of local booleans; an `HttpInterceptor` surfaces backend errors as toasts. The dead socket.io stub and commented-out code are removed. + +**Tech Stack:** Python/Flask + pytest (pure status fn is unit-tested); Angular 13 + TypeScript + Material (UI verified manually via `ng serve`; karma/headless-browser tests are out of scope in CI here). **The Angular 13→current framework upgrade remains a separate deferred follow-up — this phase does functional UI work on Angular 13 as-is.** + +**Scope:** Implements Q6 (authoritative `/status`, UI truth, error surfacing, telemetry readouts), wires UI to the Phase 3 focus/tracking routes, and the cleanup pass. Out of scope: framework upgrade. + +**Depends on:** Phases 1–3 (singletons, telemetry, recorder, focus/tracking routes). Same branch. + +--- + +## File Structure + +Backend paths relative to `WormSpy/backend/code/`; frontend paths relative to `WormSpy/wormspy/`. + +- Modify: `core/recording.py` — add `frames_written` + `started_at`/`elapsed_s` to `Recorder`. +- Create: `core/status.py` — pure `build_status(...)`. +- Create: `tests/test_status.py`; extend `tests/test_recording.py`. +- Modify: `app.py` — `/status` route; remove dead globals (`heatmap_enable` typo etc.); reconcile. +- Create: `src/app/status/status.service.ts`, `src/app/core/error.interceptor.ts`, `src/app/core/toast.service.ts` (Angular). +- Modify: `src/app/live-feed/live-feed.component.ts` + `.html` — bind to status; Hold Focus controls; readouts; poll `hist_max`; remove dead code. +- Delete: `src/app/socket/socket.service.ts` (+ its spec) — abandoned stub. +- Modify: `ReadMe.md` — reconcile codec/feature claims with the refactor. + +--- + +### Task 1: Recorder counters (frames written + elapsed) + +**Files:** +- Modify: `WormSpy/backend/code/core/recording.py` +- Test: `WormSpy/backend/code/tests/test_recording.py` (extend) + +- [ ] **Step 1: Write the failing test** + +Append to `tests/test_recording.py`: + +```python +def test_recorder_counts_frames_written(): + rec, _, _, _, _ = _recorder_with_fakes() + rec.start_recording(config={"name": "p"}) + rec.consume(_pair(0)) + rec.consume(_pair(1)) + assert rec.frames_written == 2 + rec.stop_recording() + + +def test_recorder_frames_written_resets_on_new_recording(): + rec, _, _, _, _ = _recorder_with_fakes() + rec.start_recording(config={"name": "p"}) + rec.consume(_pair(0)) + rec.stop_recording() + rec.start_recording(config={"name": "p2"}) + assert rec.frames_written == 0 + + +def test_recorder_elapsed_is_zero_when_not_recording(): + rec, _, _, _, _ = _recorder_with_fakes() + assert rec.elapsed_s == 0.0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py -k "frames_written or elapsed"` +Expected: FAIL — `AttributeError: 'Recorder' object has no attribute 'frames_written'` + +- [ ] **Step 3: Implement** + +In `core/recording.py`, add `import time` (top), and in `Recorder.__init__` add: + +```python + self.frames_written = 0 + self._started_at: Optional[float] = None +``` + +In `start_recording`, after `self._recording = True`, add: + +```python + self.frames_written = 0 + self._started_at = time.monotonic() +``` + +In `consume`, increment after a row is written (end of the method): + +```python + self.frames_written += 1 +``` + +In `stop_recording`, after `self._recording = False`, add: + +```python + self._started_at = None +``` + +Add a property: + +```python + @property + def elapsed_s(self) -> float: + return 0.0 if self._started_at is None else time.monotonic() - self._started_at +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_recording.py` +Expected: PASS (all recording tests incl. 3 new) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/recording.py WormSpy/backend/code/tests/test_recording.py +git commit -m "feat: track frames_written and elapsed time in Recorder" +``` +End commit message with: `Co-Authored-By: Claude Opus 4.8 ` + +--- + +### Task 2: Pure build_status + +**Files:** +- Create: `WormSpy/backend/code/core/status.py` +- Test: `WormSpy/backend/code/tests/test_status.py` + +> Pure assembly of the authoritative state dict so it is unit-testable without Flask/hardware. + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_status.py`: + +```python +from core.status import build_status +from core.telemetry import TelemetrySnapshot + + +def _snap(**kw): + base = dict(x=1.0, y=2.0, z=3.0, tracking_mode=0, is_tracking=False, limit_hit=False) + base.update(kw) + return TelemetrySnapshot(**base) + + +def test_build_status_reports_core_flags(): + s = build_status(snapshot=_snap(is_tracking=True, tracking_mode=1), + is_recording=True, frames_written=42, elapsed_s=4.0, + engine_running=True, zaber_connected=True, + manual_enabled=False, has_focus=True, drift_on=False) + assert s["is_recording"] is True + assert s["frames_written"] == 42 + assert s["recording_elapsed_s"] == 4.0 + assert s["is_tracking"] is True + assert s["tracking_mode"] == 1 + assert s["engine_running"] is True + assert s["zaber_connected"] is True + assert s["has_focus"] is True + assert s["x_pos"] == 1.0 and s["y_pos"] == 2.0 and s["z_pos"] == 3.0 + + +def test_build_status_passes_through_limit_hit(): + s = build_status(snapshot=_snap(limit_hit=True), + is_recording=False, frames_written=0, elapsed_s=0.0, + engine_running=False, zaber_connected=False, + manual_enabled=False, has_focus=False, drift_on=False) + assert s["limit_hit"] is True + + +def test_build_status_is_json_serializable(): + import json + s = build_status(snapshot=_snap(), is_recording=False, frames_written=0, elapsed_s=0.0, + engine_running=False, zaber_connected=False, + manual_enabled=False, has_focus=False, drift_on=False) + json.dumps(s) # must not raise +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_status.py` +Expected: FAIL — `ModuleNotFoundError: No module named 'core.status'` + +- [ ] **Step 3: Implement** + +Create `core/status.py`: + +```python +from core.telemetry import TelemetrySnapshot + + +def build_status(snapshot: TelemetrySnapshot, is_recording: bool, frames_written: int, + elapsed_s: float, engine_running: bool, zaber_connected: bool, + manual_enabled: bool, has_focus: bool, drift_on: bool) -> dict: + """Assemble the authoritative UI state. Pure: all inputs supplied by the caller (route).""" + return { + "engine_running": bool(engine_running), + "zaber_connected": bool(zaber_connected), + "is_recording": bool(is_recording), + "recording_elapsed_s": round(float(elapsed_s), 2), + "frames_written": int(frames_written), + "is_tracking": bool(snapshot.is_tracking), + "tracking_mode": int(snapshot.tracking_mode), + "manual_enabled": bool(manual_enabled), + "has_focus": bool(has_focus), + "drift_on": bool(drift_on), + "limit_hit": bool(snapshot.limit_hit), + "x_pos": float(snapshot.x), + "y_pos": float(snapshot.y), + "z_pos": float(snapshot.z), + } +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/test_status.py` +Expected: PASS (3 passed) + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/core/status.py WormSpy/backend/code/tests/test_status.py +git commit -m "feat: add pure build_status assembler" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 3: /status route + +**Files:** +- Modify: `WormSpy/backend/code/app.py` + +- [ ] **Step 1: Add the route** + +Add near the other routes in `app.py`: + +```python +from core.status import build_status + +@cross_origin() +@app.route("/status") +def status(): + return jsonify(build_status( + snapshot=TELEMETRY.snapshot(), + is_recording=RECORDER.is_recording, + frames_written=RECORDER.frames_written, + elapsed_s=RECORDER.elapsed_s, + engine_running=(ENGINE is not None), + zaber_connected=STAGE.is_connected(), + manual_enabled=isManualEnabled, + has_focus=FOCUS.has_focus, + drift_on=_drift["on"], + )) +``` + +- [ ] **Step 2: Verify** + +Parse: `python -c "import ast; ast.parse(open('WormSpy/backend/code/app.py').read()); print('parse ok')"` → `parse ok`. +Suite: `cd WormSpy/backend/code && PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q` → all pass. +On the rig (or with NullStage locally): `curl localhost:5000/status` returns the JSON with all keys. + +- [ ] **Step 3: Commit** + +```bash +git add WormSpy/backend/code/app.py +git commit -m "feat: add authoritative /status endpoint" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 4: Angular StatusService + error interceptor + toast + +**Files:** +- Create: `WormSpy/wormspy/src/app/status/status.service.ts` +- Create: `WormSpy/wormspy/src/app/core/toast.service.ts` +- Create: `WormSpy/wormspy/src/app/core/error.interceptor.ts` +- Modify: `WormSpy/wormspy/src/app/app.module.ts` (provide interceptor + MatSnackBar) + +> Angular work is verified manually with `npm install` + `ng serve` (no headless-browser test run in +> this environment). Provide the code exactly; the implementer runs the build. + +- [ ] **Step 1: StatusService (polls /status ~2 Hz)** + +Create `src/app/status/status.service.ts`: + +```typescript +import { Injectable } from '@angular/core'; +import { HttpClient } from '@angular/common/http'; +import { Observable, timer, Subject, EMPTY } from 'rxjs'; +import { switchMap, catchError, shareReplay } from 'rxjs/operators'; + +export interface BackendStatus { + engine_running: boolean; zaber_connected: boolean; + is_recording: boolean; recording_elapsed_s: number; frames_written: number; + is_tracking: boolean; tracking_mode: number; manual_enabled: boolean; + has_focus: boolean; drift_on: boolean; limit_hit: boolean; + x_pos: number; y_pos: number; z_pos: number; +} + +@Injectable({ providedIn: 'root' }) +export class StatusService { + private apiUrl = 'http://127.0.0.1:5000'; + public status$: Observable = timer(0, 500).pipe( + switchMap(() => + this.http.get(this.apiUrl + '/status').pipe(catchError(() => EMPTY)) + ), + shareReplay(1) + ); + constructor(private http: HttpClient) {} +} +``` + +- [ ] **Step 2: ToastService + error interceptor** + +Create `src/app/core/toast.service.ts`: + +```typescript +import { Injectable } from '@angular/core'; +import { MatSnackBar } from '@angular/material/snack-bar'; + +@Injectable({ providedIn: 'root' }) +export class ToastService { + constructor(private snack: MatSnackBar) {} + error(message: string): void { + this.snack.open(message, 'Dismiss', { duration: 5000, panelClass: 'toast-error' }); + } +} +``` + +Create `src/app/core/error.interceptor.ts`: + +```typescript +import { Injectable } from '@angular/core'; +import { HttpInterceptor, HttpHandler, HttpRequest, HttpErrorResponse } from '@angular/common/http'; +import { Observable, throwError } from 'rxjs'; +import { catchError } from 'rxjs/operators'; +import { ToastService } from './toast.service'; + +@Injectable() +export class ErrorInterceptor implements HttpInterceptor { + constructor(private toast: ToastService) {} + intercept(req: HttpRequest, next: HttpHandler): Observable { + return next.handle(req).pipe( + catchError((err: HttpErrorResponse) => { + // Don't toast the high-frequency /status poll failures. + if (!req.url.endsWith('/status')) { + this.toast.error(`Request failed: ${req.url.split('/').pop()} (${err.status})`); + } + return throwError(() => err); + }) + ); + } +} +``` + +- [ ] **Step 3: Register in app.module.ts** + +In `src/app/app.module.ts`, add to imports `MatSnackBarModule` (from `@angular/material/snack-bar`) +and to providers: + +```typescript +{ provide: HTTP_INTERCEPTORS, useClass: ErrorInterceptor, multi: true }, +``` + +(import `HTTP_INTERCEPTORS` from `@angular/common/http` and `ErrorInterceptor` from +`./core/error.interceptor`.) + +- [ ] **Step 4: Build check (manual)** + +Run from `WormSpy/wormspy`: `npm install` then `ng build`. Expected: build succeeds. + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/wormspy/src/app/status WormSpy/wormspy/src/app/core WormSpy/wormspy/src/app/app.module.ts +git commit -m "feat: add StatusService polling and global HTTP error toasts" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 5: Bind the UI to backend truth + Hold Focus controls + readouts + +**Files:** +- Modify: `WormSpy/wormspy/src/app/live-feed/live-feed.component.ts` +- Modify: `WormSpy/wormspy/src/app/live-feed/live-feed.component.html` +- Delete: `WormSpy/wormspy/src/app/socket/socket.service.ts` and `socket.service.spec.ts` + +> The big UX fix: toggles reflect `status$` (so refresh re-syncs); add recording timer / frame +> counter / z readout / limit-hit warning; rename FocusLock→Hold Focus and add Set Focus / Return to +> Focus / drift-comp controls calling the Phase 3 routes; poll `hist_max` via `status$`/its own timer; +> remove dead commented code and the abandoned socket service. Verified manually via `ng serve`. + +- [ ] **Step 1: Inject StatusService and expose status$ in the component** + +In `live-feed.component.ts`: inject `StatusService` in the constructor, expose `public status$ = this.sockOrStatus.status$;` (rename field as you like), and DELETE the commented-out socket code and the unused `SocketService` injection. Derive button labels from the latest status where they currently use local booleans (subscribe in `ngOnInit` to keep `isRecording`, `isTrackingEnabled`, `manualEnabled`, etc. in sync with the backend; keep optimistic local toggles for responsiveness but reconcile from `status$`). + +```typescript +import { StatusService, BackendStatus } from '../status/status.service'; +// ... +export class LiveFeedComponent implements OnInit { + public latest: BackendStatus | null = null; + constructor(private http: HttpClient, private statusSvc: StatusService) {} + ngOnInit(): void { + this.statusSvc.status$.subscribe((s) => { + this.latest = s; + this.isRecording = s.is_recording; + this.isTrackingEnabled = s.is_tracking; + this.manualEnabled = s.manual_enabled; + this.isHoldFocusSet = s.has_focus; + this.driftOn = s.drift_on; + }); + } + // Hold Focus controls + public setFocus(): void { + this.http.post(this.apiUrl + '/set_focus', {}).subscribe(() => {}); + } + public returnToFocus(): void { + this.http.post(this.apiUrl + '/return_to_focus', {}).subscribe(() => {}); + } + public toggleDriftComp(): void { + this.driftOn = !this.driftOn; + this.http.post(this.apiUrl + '/toggle_drift_comp', + { drift_enabled: this.driftOn ? 'True' : 'False' }).subscribe(() => {}); + } +} +``` + +Add the fields `isHoldFocusSet = false;` and `driftOn = false;`. Remove the old `toggleAutofocus()` +method and `isAutofocusEnabled` field. + +- [ ] **Step 2: Update the template** + +In `live-feed.component.html`: +- Replace the FocusLock toggle with three controls: "Set Focus" → `setFocus()`, "Return to Focus" → + `returnToFocus()`, and a toggle "Drift Comp (experimental)" → `toggleDriftComp()` showing `driftOn`. +- Add a status strip (visible when `latest as s`) showing: recording timer + `{{ s.recording_elapsed_s }}s`, `{{ s.frames_written }} frames`, `z = {{ s.z_pos | number:'1.0-0' }}`, + connection chips (`engine_running`, `zaber_connected`), and a warning banner when `s.limit_hit` + ("Stage limit reached — worm may be off-frame"). +- Bind the Start/Stop Recording and Tracking button labels to `latest?.is_recording` / + `latest?.is_tracking` so they reflect the backend after a refresh. + +```html +
+ engine + zaber + REC {{ s.recording_elapsed_s }}s · {{ s.frames_written }} frames + z = {{ s.z_pos | number:'1.0-0' }} + ⚠ Stage limit reached +
+``` + +- [ ] **Step 3: Delete the abandoned socket stub** + +```bash +git rm WormSpy/wormspy/src/app/socket/socket.service.ts WormSpy/wormspy/src/app/socket/socket.service.spec.ts +``` + +Remove any remaining imports of `SocketService`. + +- [ ] **Step 4: Build check (manual)** + +From `WormSpy/wormspy`: `ng build`. Then `ng serve` and, with the backend running, verify: toggles +reflect backend state across a page refresh; Set/Return Focus call through; recording shows a live +timer + frame count; the limit warning appears when tracking hits a limit; a failed request shows a +toast. + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/wormspy/src/app/live-feed +git commit -m "feat: bind UI to backend /status; add Hold Focus controls and telemetry readouts" +``` +End commit message with the Co-Authored-By trailer. + +--- + +### Task 6: Backend cleanup + README reconciliation + +**Files:** +- Modify: `WormSpy/backend/code/app.py` +- Modify: `ReadMe.md` + +> Remove cruft exposed by the refactor; reconcile docs. No behaviour change. + +- [ ] **Step 1: Remove dead globals / fix the typo'd one** + +In `app.py`: remove the `heatmap_enable` typo global (keep `heatmap_enabled` if heatmap is still +wired, else remove both and the `/toggle_heatmap` route if unused). Remove now-unused module globals +left from the old generators (`start_recording`, `stop_recording`, `start_recording_r`, +`stop_recording_r`, `hist_frame`/`hist_max` if the histogram is reworked, `serialPort` if unused, +`nodeIndex` if DLC node is set elsewhere). For each, confirm with `grep -n app.py` that it has +no remaining readers before deleting. + +- [ ] **Step 2: Decide BaslerCamera disposition** + +`grep -n "BaslerCamera" app.py` — it is a legacy alt-camera wrapper now superseded by +`core/cameras_spinnaker.py`. If unused, remove it from `app.py` (a future `core/cameras_basler.py` +adapter can be added later if needed). If you keep it, leave a comment that the canonical path is the +`core` adapters. + +- [ ] **Step 3: Reconcile the README** + +In `ReadMe.md`, update the recording description: brightfield is now an **intra-frame codec (MJPG, +or FFV1 lossless)** — not XVID/“MJPG-via-fourcc-XVID” — and the right camera remains 16-bit TIFF. +Update the autofocus/“FocusLock” mention to **Hold Focus** (operator-set; optional drift comp). +Add a one-line note that recordings now emit a per-frame CSV spine keyed by `frame_index`. + +- [ ] **Step 4: Verify** + +Parse: `python -c "import ast; ast.parse(open('WormSpy/backend/code/app.py').read()); print('parse ok')"` → `parse ok`. +Suite: `cd WormSpy/backend/code && PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q` → all pass. + +- [ ] **Step 5: Commit** + +```bash +git add WormSpy/backend/code/app.py ReadMe.md +git commit -m "chore: remove dead globals/BaslerCamera; reconcile README with refactor" +``` +End commit message with the Co-Authored-By trailer. + +--- + +## Self-Review + +**Spec coverage (Q6):** +- *Authoritative state via polled /status* → Tasks 2 (`build_status`) + 3 (route) + 4 (`StatusService` ~2 Hz). ✓ +- *UI reflects backend truth (fixes refresh-lie)* → Task 5 Step 1 (`status$` reconciles toggles). ✓ +- *Surface HTTP errors* → Task 4 (`ErrorInterceptor` → toast; skips the /status poll). ✓ +- *Telemetry readouts (recording timer, frame counter, z, warnings)* → Tasks 1 (counters) + 5 Step 2 (status strip). ✓ +- *Hold Focus controls wired to Phase 3 routes* → Task 5 (set/return/drift). ✓ +- *Poll hist_max instead of one-shot* → Task 5 (status strip / dedicated timer; the legacy one-shot `callHistMax` is removed). ✓ +- *Delete dead socket.io stub + commented code* → Task 5 Step 3 (rm socket service) + Step 1 (remove commented socket code). ✓ +- *Cleanup dead backend code + reconcile README* → Task 6. ✓ + +**Placeholder scan:** Backend tasks (1–3, 6) carry complete code + tests/parse checks. Angular tasks (4–5) carry complete code with explicit manual `ng build`/`ng serve` verification (no headless-browser CI here) — this is the standard "implement + manual verify" pattern for UI, not a placeholder. + +**Type consistency:** `BackendStatus` (TS) mirrors `build_status` (py) keys exactly (Tasks 2/4). `Recorder.frames_written`/`elapsed_s`/`is_recording` used in Tasks 1/3. `StatusService.status$: Observable` consumed in Task 5. The Phase 3 routes `/set_focus`, `/return_to_focus`, `/toggle_drift_comp` called in Task 5 match Phase 3 definitions. + +**Known follow-ups (not gaps):** the Angular 13→current framework upgrade (deferred); karma/headless-browser unit tests for the new Angular services (the harness here can't run them); a richer histogram view if the one-shot `get_hist` stream is reworked. From 9d56dbf8cb0e92ef2385f187f779d77ec9c1d3fc Mon Sep 17 00:00:00 2001 From: Sebastian W Date: Sun, 21 Jun 2026 10:44:41 -0400 Subject: [PATCH 13/13] docs: note accepted Phase 1 regression and insertion-order fix in plan Co-Authored-By: Claude Opus 4.8 --- .../plans/2026-06-21-acquisition-core-refactor.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md b/docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md index 1335da0..f4cd1e2 100644 --- a/docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md +++ b/docs/superpowers/plans/2026-06-21-acquisition-core-refactor.md @@ -1186,7 +1186,11 @@ git commit -m "feat: add SpinnakerCamera adapter configured as the acquisition c - Add startup construction near `app = Flask(...)` (`app.py:24-27`). - Repoint `move_to_center` (`app.py:454-463`) and the manual controller (`app.py:656-749`) at the shared `STAGE`. -> **Note:** This task is hardware-wired and validated by launching the app on the rig (Step 5). It removes acquisition from the HTTP handlers (the root cause of loop drift and motor-on-disconnect). Recording and Hold Focus are intentionally left on their existing code paths for now (Phases 2–3); the engine `sink` is wired but set to a no-op placeholder so behaviour is unchanged until Phase 2. +> **Note:** This task is hardware-wired and validated by launching the app on the rig (Step 5). It removes acquisition from the HTTP handlers (the root cause of loop drift and motor-on-disconnect). The engine `sink` is wired but set to a no-op placeholder. +> +> **ACCEPTED TEMPORARY REGRESSION (decided 2026-06-21):** Because the legacy recording loop lived *inside* the old `video_feed` generators and autofocus/manual/tracking depended on globals those generators set, this minimal Task 9 leaves the app with working **dual display + Center** only. Recording is a silent no-op; `toggle_af` and `toggle_manual` will return HTTP 500 (NameError on the removed motor globals); `toggle_tracking` does nothing (no consumer). These are restored in Phases 2–3 (recording/codec, Hold Focus, and wiring the `Tracker` with the ported algorithms). Do **not** patch the orphaned code in this task. +> +> **Insertion-order correction:** place the startup wiring block (Step 1) *after* the configuration constants (after the `settings = {...}` dict / `FPS`), NOT after the CORS block — `STAGE = connect_stage(XYmotorport, Zmotorport)` executes at import time and needs `XYmotorport`/`Zmotorport`/`FPS` already defined. - [ ] **Step 1: Add startup wiring after the Flask app is created**