Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions anvil/backends/jetson.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
local GPU stack to the robot.

Train still happens on forge/lab; export PEFT/GGUF, then sample here.
Do **not** write Anvil ``run_dir`` / media CAS / training logs on the edge
device — robotics owns disk policy; free space is scarce.

Environment
-----------
Expand Down
11 changes: 8 additions & 3 deletions anvil/data/robot_pack.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
"""In-house robot episode packs (j30 / edge vision loop → Anvil).

The Orin-class house robot logs **frames + optional detections/captions**
The Orin-class house robot may produce **frames + optional detections/captions**
and, when teleop or scripted motion exists, **action vectors**. This module
turns a small on-disk pack into :class:`~anvil.protocol.trajectory.Trajectory`
objects and Anvil JSONL for ``run_robot_offline`` / VLM SFT.
turns a small **lab-side** on-disk pack into
:class:`~anvil.protocol.trajectory.Trajectory` objects and Anvil JSONL for
``run_robot_offline`` / VLM SFT.

**Device ops:** edge robots (j30) are owned by the **robotics** project. Packs
are assembled **off-device** after a capped pull. Do not treat the robot as an
Anvil log sink — free disk and RAM are scarce; prefer few low-res frames.

Pack layout (episode pack, Bridge-like intermediate)::

Expand Down
28 changes: 23 additions & 5 deletions docs/datasets-robotics.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,18 +161,36 @@ Python: `anvil.data.robot_pack.house_pack_to_trajectories` / `house_pack_to_json

#### j30 `vision/out` → house pack

If the robot already logs under `~/vision/out` (llm RGB+see text, loop
escalations, live_pull frames + captions.json):
**Ops ownership:** the reComputer j30 (and `~/vision`) is run by the
**robotics** project. Anvil does not own on-device logging policy.

**Storage:** Orin Nano-class devices fill up fast if frame dumps are left
running. Anvil’s contract is **pull a small snapshot → convert on lab → train
on lab**. Do not use the robot as an Anvil run dir, HF cache, or multi-hour
JPEG ring buffer unless robotics explicitly budgets the space.

If a **short** capture already exists under `~/vision/out` (llm RGB+see text,
loop escalations, optional live_pull):

```bash
# operator: rsync vision/out from the robot to a *local* path (no secrets in git)
rsync -avz <robot-user>@<robot-host>:~/vision/out/ ./j30-out/
# lab machine only — no secrets/IPs in git
# Prefer size-capped rsync; prune on-device after pull (robotics policy).
rsync -avz --max-size=5m <robot-user>@<robot-host>:~/vision/out/ ./j30-out/

python scripts/j30_vision_out_to_pack.py --source ./j30-out --out ./house_pack
python scripts/robot_pack_smoke.py --pack ./house_pack --steps 20
# train/export on forge; ship a small GGUF/adapter back — not the reverse
```

Never commit house frames or LAN host credentials.
**Do not:**

- Enable continuous high-rate frame logging “for Anvil” without a retention
cap (count, MB, or TTL) owned by robotics
- Leave Anvil `run_dir` / media CAS / model weights on the j30
- Commit house frames or LAN credentials

Default capture style for packs: **low res** (e.g. 320×240), **few frames per
episode**, text captions/detections over raw video.

## Product goals

Expand Down
13 changes: 13 additions & 0 deletions docs/edge-export.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,19 @@ export ANVIL_JETSON_MODEL=smolvlm-256m

CI uses `JetsonSampleConfig(dry_run=True)`.

## Edge device storage (j30 / Orin)

| Do on robot | Do on lab (forge / Mac) |
|-------------|-------------------------|
| Serve small GGUF (smol ~256M) | Train LoRA, build packs, write `run_dir` |
| Short vision bursts under **robotics** retention | Hold episode packs + media CAS |
| Prune `~/vision/out` after pull | Convert `vision/out` → house pack |

The j30 is **ops-owned by robotics**, not Anvil. Anvil agents must not invent
long-running capture jobs or leave multi-GB logs on device. SDKs (Orbbec,
etc.) already dominate disk; treat free space as scarce even when `df` looks
comfortable.

## Smoke

```bash
Expand Down
1 change: 1 addition & 0 deletions docs/handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Use this when starting a **new agent session** dedicated to Anvil (separate from
**Habit:** `python scripts/lab_smokes.py --profile quick` often.
**Resume:** SFT/VLM/GRPO/DPO all support `checkpoint_every` + `resume=True`.
**Robot:** `scripts/robot_pack_smoke.py` · edge: `docs/edge-export.md` · agent: `scripts/agent_dogfood.py`.
**j30 ops:** robotics project owns the device; Anvil = pull-off → pack → lab train; respect storage (no log dump on Orin).
**Multi-worker sample:** `docs/multi-worker.md`

**Recommended next Outcome:**
Expand Down
1 change: 1 addition & 0 deletions docs/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Workers resolve that id via lab paths / HF cache on the train host — clients d
- Prefer **SmolVLM-256M** (or 500M) onboard when RAM is tight; use **3B** on forge as teacher if you distill later.
- Edge export: PEFT/merged → AWQ / ONNX / TRT (Phase 4.C).
- Same Anvil multimodal message schema lab ↔ edge; action targets are **text tokens** (bins).
- **Storage:** do not keep Anvil train runs, multi-GB media, or HF caches on the robot. Device logging policy is **robotics-owned**; Anvil only consumes pulled snapshots (see `docs/datasets-robotics.md`, `docs/edge-export.md`).

## Optional lab bases

Expand Down
20 changes: 17 additions & 3 deletions scripts/j30_vision_out_to_pack.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,25 @@
#!/usr/bin/env python3
"""Convert a local j30 ``vision/out`` tree → Anvil house episode pack.
"""Convert a **local** j30 ``vision/out`` tree → Anvil house episode pack.

Does **not** SSH or store host credentials. Operator pulls frames first::
**Ownership / storage (read this):**

rsync -avz user@jetson:~/vision/out/ ./j30-out/
- The j30 (and on-device ``~/vision``) is under **robotics project operational
control**, not Anvil. Anvil only consumes **already-pulled** snapshots.
- The Orin is **storage- and RAM-constrained**. Logging frames is expensive.
Prefer short bursts → rsync **off-device** → prune on the robot. Do **not**
leave multi-GB capture trees, train run dirs, or HF caches on the device.
- This script never SSHes, never writes to the robot, and never stores
credentials. Host IPs/passwords must not enter the Anvil git tree.

Operator flow (lab Mac / forge — not on the robot)::

# 1) pull a *small* snapshot (robotics ops decides retention on device)
rsync -avz --max-size=5m user@jetson:~/vision/out/ ./j30-out/
# 2) convert locally
python scripts/j30_vision_out_to_pack.py --source ./j30-out --out ./house_pack
# 3) train offline on lab, not on j30
python scripts/robot_pack_smoke.py --pack ./house_pack --steps 20
# 4) optional: after pull, prune on-device out/ per robotics policy

Expected source layout (subset OK)::

Expand Down
1 change: 1 addition & 0 deletions start.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Not a full MoE train stack on day one.
- Prefer **small PRs**; design / product thesis changes go through `docs/` before large code.
- Do not brand Anvil with third-party product names or claim proprietary API compatibility.
- Vision and robot/Jetson paths stay **first-class in design**; don’t strip them to ship text-only forever.
- **Edge device storage** — house robots (e.g. j30) are **ops-owned by the robotics project**, not Anvil. Prefer **pull-off → train on lab**; do not treat the robot as a log/training dump. Cap and prune on-device captures; never commit device blobs or LAN secrets.
- Agent **force** past architecture gates must stay **audited**.

## Facts (one screen)
Expand Down
Loading