From a83201d28183866abb50054ca41d7b3893ae2529 Mon Sep 17 00:00:00 2001 From: William Wong Date: Mon, 13 Jul 2026 13:33:59 +0100 Subject: [PATCH] =?UTF-8?q?docs:=20v0=20wrap-up=20=E2=80=94=20the=20stub?= =?UTF-8?q?=20inventory=20and=20the=20usage=20guide?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit docs/v0-stub.md — every stand-in in the system, in one place, so nothing in a screenshot can be mistaken for a finding. The five fixtures; the prompt hash; the triptych's model signature; the ghost-cloud perturbation; the identity rotations. Plus what is NOT fake and survives v1 untouched. Two things fell out of writing it down: - NOT ONE of the five authored phrases returns the fixture it was written for. Type "a body remembers a place it cannot return to" — the project's own signature phrase — and you get a body sliding sideways. That is what "the system does not understand what you type" looks like written down honestly. - sum(ord(c)) for "snapmogen" and "kimodo" are congruent mod 5, so those two triptych panels ALWAYS select the same base fixture. Two thirds of the triptych's visible difference is seeded noise on one identical motion. docs/usage.md — the guide: running it, every view, every control, the full keyboard table (they were undiscoverable), the boot flags, how to read each of the four registers, the fixture-authoring trap, and how to swap the backend. fix(app): T outside performance mode jumped straight to 0.25x. The tempo index was seeded for performance mode's opening state (0.5x) rather than the bench's (1x), so the first press skipped a step. Found while documenting it. Co-Authored-By: Claude Opus 4.8 --- README.md | 21 +++- docs/usage.md | 242 +++++++++++++++++++++++++++++++++++++++ docs/v0-stub.md | 159 +++++++++++++++++++++++++ frontend/app/src/main.ts | 10 +- 4 files changed, 425 insertions(+), 7 deletions(-) create mode 100644 docs/usage.md create mode 100644 docs/v0-stub.md diff --git a/README.md b/README.md index 0e984a2..5c5b8db 100644 --- a/README.md +++ b/README.md @@ -203,11 +203,14 @@ spine of it (everything except the models): - ✗ **A model behind the service** — the v1 step; needs weights + likely a GPU. *Not built.* ``` -fixtures/ canonical motion JSON (hand-authored) + generator +fixtures/ canonical motion JSON (hand-authored) + generator +docs/abstract.md the accepted abstract — the canonical framing docs/motion-schema.md the exchange-format spec -service/ FastAPI /generate stub (uv) -frontend/app/ Vite + three.js Lab Bench (the live screen) -frontend/mockups/ the original static mockups (reference) +docs/usage.md how to use the tool — every view, control and shortcut +docs/v0-stub.md what v0 fakes — the complete honesty inventory +service/ FastAPI /generate stub (uv) +frontend/app/ Vite + three.js Lab Bench (the live screen) +frontend/mockups/ the original static mockups (reference) ``` **Stack:** three.js + TypeScript + Vite (frontend, pnpm); Python + FastAPI (service, uv); @@ -236,6 +239,9 @@ Open , type a phrase, click **Generate** — a 3D stick f animates; drag to orbit, use play/pause and the scrub bar. To re-author the motions, edit and re-run `python3 fixtures/_generate.py`. +📖 **[`docs/usage.md`](docs/usage.md) is the full guide** — every view, every control, every +keyboard shortcut, and how to read each of the four notation registers. + ### Reading it Hit **Read** (or press R) for the four **notation registers** — the same motion @@ -301,6 +307,13 @@ keeping its native way of authoring) and **performance mode** (the projectable s **They are not three models interpreting a theme.** Nothing in that view can be read as a finding about model behaviour. +> **There are five hand-authored motions in the entire system.** Every movement you have ever +> seen BodyPrompt produce is one of those five, wearing a little seeded jitter. + +⚠️ **[`docs/v0-stub.md`](docs/v0-stub.md) is the complete inventory of what v0 fakes** — every +stand-in, written down in one place, so that nothing in a screenshot can be mistaken for a +finding. Read it before citing anything this tool shows you. + Everything above is the research **instrument** — deliberately built first, so that v1 only has to swap the stub for a model and every one of these views becomes real at once. Repo: **Public**. diff --git a/docs/usage.md b/docs/usage.md new file mode 100644 index 0000000..2937c5a --- /dev/null +++ b/docs/usage.md @@ -0,0 +1,242 @@ +# Using BodyPrompt + +The complete guide to the running app: every view, every control, every keyboard shortcut. + +**Before you start, one thing.** The app does not read your prompt. It hashes it, and returns +one of five hand-authored motions. Everything below describes a working *instrument* pointed +at *fake data* — see [`v0-stub.md`](v0-stub.md) for exactly what is faked and where. Nothing +you see in this tool is yet a finding about a model. + +--- + +## Running it + +Two processes. Needs **Python 3.10+ with [uv](https://docs.astral.sh/uv/)** and **Node 18+ +with [pnpm](https://pnpm.io/)**. + +```bash +# 1) the service — serves motions on http://localhost:8000 +cd service +uv run uvicorn app.main:app --port 8000 + +# 2) the app — http://localhost:5173 (second terminal) +cd frontend/app +pnpm install +pnpm dev +``` + +Check the service is alive: `curl localhost:8000/health` → +`{"ok":true,"backend":"stub","ml":false,"ready":true}`. The `"ml":false` is the tell — no +model is running. + +If the app can't reach the service it says so on the stage rather than failing silently, and +prints the command to start it. + +--- + +## Keyboard shortcuts + +Every shortcut, in one table. **Shortcuts are ignored while your cursor is in the prompt box** +— otherwise typing the letter "p" would drop you into performance mode mid-sentence. The one +exception is esc, which always works, because you do not want to be hunting for a +mouse in front of an audience. + +| Key | Does | +|---|---| +| R | **Read** — open / close the four notation registers | +| C | **Compare** — open / close the multi-model triptych | +| P | **Perform** — enter / leave performance mode | +| space | play / pause | +| T | cycle tempo: 0.5× → 0.25× → 1× | +| G | ghost-cloud on / off | +| esc | leave the current mode; from the prompt box, unfocus it | +| enter | (in the prompt box) generate | + +**Mouse, on the 3D stage:** drag to orbit, scroll to zoom, right-drag to pan. Standard +three.js orbit controls. + +## Boot flags + +Append to the URL. Useful for projectors and for scripted screenshots — you don't want to be +clicking through chrome in front of a room. + +| URL | Opens | +|---|---| +| `localhost:5173/` | the Lab Bench | +| `localhost:5173/?registers=1` | straight into the four notation registers | +| `localhost:5173/?compare=1` | straight into the triptych | +| `localhost:5173/?perform=1` | straight into the projectable performance stage | + +--- + +## The Lab Bench (the default view) + +Type a phrase, press enter or click **Generate**. A 3D stick figure plays the +returned motion. + +**Deliberately not a realistic avatar.** A realistic body sells an illusion and invites you to +read a *character*; a stick figure exposes the computational body directly — joints, +trajectories, timing, weight. See the README on why. + +Around the stage: + +- **Telemetry** (top left) — model, prompt, seed, joint count, how many other seeds are in the + cloud, and an amber `stub · hand-authored fixture (no ML)` line. That last line is there so + that no screenshot of this app can be honestly mistaken for model output. +- **Lineage rail** (left) — the search so far. See below. +- **Notation rail** (right) — two of the four registers, small: the notation strip and the + floor path. R opens all four, large. +- **Transport** (bottom) — play/pause, a scrub bar, the ghost-cloud toggle, and a frame + counter. Scrubbing moves the figure, the ghosts, and every open register together. + +### The prompt lineage tree — the core contribution + +**Every prompt revision is kept.** Generating does not replace what came before; it adds a +child node. The search branches, and nothing is undone. + +- **Generate** from the current node → extends the line. +- **Click a past node**, then Generate → **branches** from there. The old line stays. +- Clicking a node **replays its stored motion** — no re-fetch, and it restores that node's own + ghost-cloud too. + +The tree is the artefact, not the leftovers. In performance it is the set: the audience +watches not just generated movement but the *evolution of thought* — how a phrase mutated, +where it branched, which possibility was followed and which was left open. + +### The ghost-cloud + +One prompt, many seeds. The solid amber figure is the primary motion; the translucent blue +figures around it are three siblings with different seeds. Toggle with G or the +checkbox. + +Where the lineage tree shows *the search across time*, the ghost-cloud shows *the possibilities +at one moment*. It is hidden in the triptych and the registers view, because there it would +only add noise — the ghost-cloud compares **seeds**, the triptych compares **models**. + +*(In v0 this is a seeded perturbation, not a model sampling. See [`v0-stub.md`](v0-stub.md).)* + +--- + +## The four notation registers — R + +The same motion, made legible four ways at once. A stick figure lets you *watch* movement; it +does not let you **read** it. + +**1 · Chronophotograph.** Marey's plate: seven exposures fading from past to present, so the +whole phrase is visible at once instead of streaming past. The lit pose is *now*. +The horizontal axis is **time, not distance** — as on Marey's *moving* plate. Each pose is +centred on its own pelvis, so the body's travel is dropped (otherwise a motion that slides a +metre sideways walks its last exposure clean off the plate). Seen from a quarter-turn, because +dead-on the hips are only 9 cm apart and the two legs collapse onto a single line. + +**2 · Notation strip.** A time-scored staff, one row per limb — L arm, R arm, spine, weight, +feet. Each glyph's **angle** is the direction that limb travelled, its **length** is how far, +and its **height in the row** is the level. Limbs are read *relative to their anchor* (a wrist +against its shoulder is the arm's gesture; the wrist's absolute position would just re-tell you +where the body is). "Weight" is the exception — the pelvis in absolute terms *is* where the +weight is. A bucket in which nothing moved still gets a dot, so the score never lies by +omission. + +Level and magnitude are normalised **per track**, so "high" means high *for that limb* — right +for reading one motion closely, but it means two strips are not directly comparable. + +**3 · Floor path.** The movement from above: the weight's trace, the feet faint behind it, a +marker where it began and a dot where it is now. Never zoomed in past 0.7 m — a body that +barely travels should *look* like a body that barely travels. + +**4 · Laban-inspired score.** A vertical staff read **bottom → top**. It is a **designed +reduction, not strict Labanotation** — designing that reduction is itself part of the research. + +| Element | Means | +|---|---| +| The central column pair | **support** — which foot is bearing weight. Read from the foot joint; if both feet leave the floor the column goes empty, and the *gap* is the notation. | +| Support shading | how deep the body is sitting (a crouch shades it solid) | +| Outer columns | the limb **gestures** — the body's own left and right, as the performer would read them, not as you watching would | +| Glyph **fill** | **level**: solid = low · hatched = middle · hollow = high | +| Glyph **lean** | which way it went, sideways | +| Glyph **width** | how far it went | + +Unlike the notation strip, levels here are read **anatomically, not statistically** — a hanging +wrist sits ~0.57 m below its shoulder (*low*), level with the shoulder is *middle*, above is +*high*; a standing ankle rests ~0.09 m up (*low*). That means "high" means the same thing in +every motion, so two Laban scores **can** be compared. This is a real difference in kind +between register 2 and register 4, and it is deliberate. + +### No register is complete — that is the point + +Each one throws information away, and **which** thing it throws away is the argument. The floor +path cannot show you a raised arm. The chronophotograph drops the body's travel. The Laban +score leaves forward/back to the floor path. Reading them *against* each other — and noticing +what falls *between* them — is the instrument. All four playheads walk the phrase together, so +that reading-against is possible at all. + +**Try this:** the only fixture in which an arm rises above the shoulder is `reach-and-return`, +so on most prompts the Laban score's level shading is honestly, monotonously "low". To see the +fill actually change (solid → hatched → hollow), find a prompt that lands on that fixture — with +model **snapmogen**, the phrase "slip away" does it. (Yes, really. That is fake #1 in +[`v0-stub.md`](v0-stub.md), and it is exactly as arbitrary as it sounds.) + +--- + +## The triptych — C + +One prompt, three models, side by side: SnapMoGen, Language of Motion, Kimodo. Each keeps its +**native way of authoring** — write / voice / sculpt — because the difference in *how you +author* is itself part of the research. (Only *write* is wired up; the others are labelled, not +built.) Each panel has its own accent colour, and all three play in step — comparing motions +that were out of step with each other would tell you nothing. + +> ⚠️ **The three models differ only because the stub hashes `(model, prompt)`.** They are not +> three models interpreting a theme. In fact two of the three panels usually show the *same* +> base fixture wearing different seeded jitter. There is a banner in the UI saying so, and it +> is not being coy. **Do not use a triptych screenshot as evidence of model comparison.** + +--- + +## Performance mode — P + +The projectable stage, for the lecture performance. Not a separate page: the **same session, +the same lineage.** The performer keeps working — typing, generating, branching — while the +room sees only the body, the phrase and the score. + +What changes: the instrument chrome falls away, the background darkens, a spotlight gathers the +eye onto the body, the phrase goes large beneath it, and **playback drops to half speed** — +because a human has to be able to *follow and re-embody it*. The score gets thicker strokes and +bigger labels: it has to be readable by a body, from across a room. The lineage keeps growing in +the corner — research log as set. + +T cycles the tempo, space plays and pauses, esc gets you out. +`?perform=1` boots straight into it, for plugging into a projector. + +--- + +## Changing the movement itself + +The five motions are hand-authored keyframes in +[`fixtures/_generate.py`](../fixtures/_generate.py). Edit the keyframe lists and re-compile: + +```bash +python3 fixtures/_generate.py +``` + +**One trap, and it has drawn blood.** The fixture authoring uses **translation-only forward +kinematics**: a joint's total offset is the sum of its own delta and all its ancestors'. So +**dropping the pelvis drags the legs down with it** — a crouch needs compensating `+y` knee and +ankle deltas or the feet sink through the floor. `gather` shipped with its feet 7 cm +underground before a foot-plant check caught it. If you author a crouch, check the feet. + +The service loads fixtures at startup, so restart it after re-compiling. + +## Swapping the backend + +The generator behind `POST /generate` is chosen by an environment variable: + +```bash +BODYPROMPT_BACKEND=stub uv run uvicorn app.main:app --port 8000 # the default; the only one that exists +``` + +A real backend implements `Generator` (`name`, `ml`, `ready()`, `generate(model, prompt, +variants)`), returns canonical motion, and registers itself in `_BACKENDS`. **Nothing else in +the system changes** — not the frontend, not the renderer, not one of the four registers. That +is the whole reason v0 was built in this order. See +[`motion-schema.md`](motion-schema.md) for the format a backend must emit. diff --git a/docs/v0-stub.md b/docs/v0-stub.md new file mode 100644 index 0000000..c37c7ca --- /dev/null +++ b/docs/v0-stub.md @@ -0,0 +1,159 @@ +# What v0 fakes + +**A complete inventory of every stand-in in the system, written down in one place so that +nothing in a screenshot can be mistaken for a finding.** + +v0 built the *research instruments* before the model, deliberately: the search loop, the +lineage tree, the ghost-cloud, the four notation registers, the triptych and performance +mode all run end-to-end — on hand-authored data. That was the right order to build in, but +it leaves the repo in a state where **the instrument looks like it is working when in fact +it is not reading your prompt at all.** This document says exactly where the seams are. + +The single sentence version: + +> **There are five hand-authored motions in the entire system. Every movement you have ever +> seen BodyPrompt produce is one of those five, wearing a little seeded jitter. The prompt +> is not read; it is hashed.** + +--- + +## The five fixtures + +Hand-authored keyframes in [`fixtures/_generate.py`](../fixtures/_generate.py), compiled to +canonical motion JSON. All five are 90 frames at 30 fps — exactly 3.0 seconds. + +| Fixture | Phrase it was authored *for* | Tagged model | Seed | What the body does | +|---|---|---|---|---| +| `reach-and-return` | "a body remembers a place it cannot return to" | snapmogen | 4021 | The right arm reaches up and forward, then almost — but not quite — returns. A residual 8 cm remains: the memory. **The only fixture in which an arm goes above the shoulder.** | +| `slip-away` | "slip away" | kimodo | 1177 | The whole body drifts 0.46 m sideways and turns, arms trailing behind. **The only fixture that really travels.** | +| `gather` | "coming home" | language-of-motion | 8802 | Both arms sweep inward to the chest; a slight crouch. | +| `fall-and-rise` | "the ground remembers" | snapmogen | 5310 | A fold down toward the floor (the pelvis drops 45 cm), then a rise, ending a little taller than rest — opened out. | +| `turn-away` | "look back, then go" | kimodo | 2287 | The torso twists and the head turns back over the shoulder, one arm sweeping behind; then the body unwinds and leaves. | + +The "phrase it was authored for" column is **not** a lookup table. Read on. + +--- + +## Fake #1 — the prompt is hashed, not read + +[`StubGenerator.generate`](../service/app/generators.py) picks a fixture like this: + +```python +key = f"{model}:{prompt}" +idx = (sum(ord(c) for c in key)) % len(self._fixtures) # 5 fixtures +``` + +The sum of the character codes, modulo five. That is the entire "understanding" of language +in v0. It is **stable** (the same prompt always returns the same motion, which is what makes +the lineage tree meaningful) and it is **meaningless** (the mapping carries no relation +whatever between what you typed and what the body does). + +Here is the proof, and it is sharper than I expected. These are the five phrases the fixtures +were *authored for*, run through the real hash: + +| You type… | snapmogen gives you | language-of-motion gives you | kimodo gives you | +|---|---|---|---| +| "a body remembers a place it cannot return to" | `slip-away` | `gather` | `slip-away` | +| "slip away" | `reach-and-return` | `fall-and-rise` | `reach-and-return` | +| "coming home" | `fall-and-rise` | `slip-away` | `fall-and-rise` | +| "the ground remembers" | `slip-away` | `gather` | `slip-away` | +| "look back, then go" | `turn-away` | `reach-and-return` | `turn-away` | + +**Not one phrase returns the motion it was written for.** Type "slip away" and the body +reaches for something. Type "a body remembers a place it cannot return to" — the project's +own signature phrase, the one in the abstract — and you get a body sliding sideways, which +is not what that fixture was authored to express at all. + +This is not a bug to be fixed. It is what "the system does not understand what you type" +*looks like* when you write it down honestly. It is fixed by a model, not by a better hash. + +## Fake #2 — the triptych's three models + +Two models collide. `sum(ord(c))` for `"snapmogen"` and for `"kimodo"` happen to be congruent +modulo 5 — so for **every** prompt, those two panels select the **same base fixture**. Look at +the table above: the snapmogen and kimodo columns are identical, row for row. + +What stops the triptych from showing literal twins is this, in `generate()`: + +```python +model_sig = sum(ord(c) for c in model) * 7919 if model else 0 +seed = int(base.get("seed", 0)) + model_sig +motion = vary(base, seed) if model_sig else dict(base) +``` + +Each model gets a stable signature, and the motion is jittered by it. So in a two-thirds of +the triptych, **the visible difference between "SnapMoGen" and "Kimodo" is nothing but seeded +noise applied to one identical fixture.** The remaining panel differs only because a modulo +landed elsewhere. + +The triptych carries an in-UI banner saying the differences are a stub artefact. That banner +is doing heavy lifting, and it is not overstating the case. **Nothing in that view can be read +as a finding about model behaviour.** Do not put a triptych screenshot in a paper as evidence +of models interpreting a theme. + +## Fake #3 — the ghost-cloud is a perturbation, not a sampling + +`vary(motion, seed)` gives every joint a smooth, low-frequency sinusoidal wander — amplitude, +frequency and phase drawn from a seeded RNG, amplitude scaled per joint by the `_WANDER` +table (pelvis ~1.2 cm, wrists ~7 cm; vertical wander damped to 0.6 so the figure doesn't bob +off the floor). + +That table is a *guess at the shape* of a model's variance — the root and spine barely move +while the extremities vary a lot, so the cloud reads as "the same intention, differently +expressed" rather than as noise. A real model's variance does behave roughly like that. But +this cloud is **not a model sampling different outputs.** It is one motion, wobbled five ways. + +The claim the ghost-cloud is currently entitled to make is only: *here is what a variance +display would look like, and here is the interface for reading one.* + +## Fake #4 — rotations are empty, and the FK is translation-only + +Every frame carries a `rotations` array, and in v0 it is **entirely identity quaternions** — +reserved space in the schema, filled with nothing. All motion lives in `positions`. + +The fixtures themselves are authored by summing per-joint translation offsets down the +kinematic chain (see `total_offset` in `_generate.py`) — not by rotating limbs. This has a +consequence that bit once already: **dropping the pelvis drags the legs down with it**, so +every crouch needs compensating `+y` knee and ankle deltas or the feet sink through the +floor. `gather` shipped with its feet 7 cm underground until a foot-plant check caught it. + +A real model will emit rotations. When it does, the renderer and the notation registers keep +working (they read positions), but anything that wants joint *orientation* — a facing +direction, a proper Labanotation front/back symbol — becomes possible for the first time. + +--- + +## What is **not** fake + +Everything the fakes are wearing. This is the part that survives v1 untouched, and it is why +building the instrument first was worth doing: + +- **The canonical schema** (`bodyprompt.motion/v0`) — the exchange format, and the whole + reason a model swap is a swap rather than a rewrite. → [`motion-schema.md`](motion-schema.md) +- **The `Generator` interface and the `POST /generate` contract.** A real backend implements + three methods and registers itself in `_BACKENDS`. Nothing else in the system changes. +- **The stick-figure renderer** — reads canonical motion, knows nothing of where it came from. +- **The prompt lineage tree** — the core contribution. Branching, retention, click-to-replay. + Real now, real later. +- **All four notation registers** — chronophotograph, notation strip, floor path, + Laban-inspired score. Every glyph is derived from joint trajectories. They are *real + readings of fake data*, and the moment the data is real, so are they. +- **The triptych as an instrument** (the comparison is real; the models are not). +- **Performance mode**, the transport, the tempo control, the ghost-cloud *display*. + +## Retiring the fakes + +Every one of them dies at v1, and they die together — that is the point of having built it +this way. One backend that returns joints instead of a fixture, and: + +| Fake | Retired by | +|---|---| +| #1 the prompt hash | the model reading the prompt | +| #2 the triptych's model signature | three actual models | +| #3 the ghost-cloud perturbation | asking the model for *n* samples | +| #4 identity rotations | a model that emits them | + +The one thing v1 must not do is return **rendered video**. Several hosted motion-generation +APIs do exactly that, and a video cannot be fed to the renderer, the ghost-cloud, or any of +the four registers — it would throw away the entire instrument this repo is. **v1 needs +joints.** diff --git a/frontend/app/src/main.ts b/frontend/app/src/main.ts index 025237a..227492a 100644 --- a/frontend/app/src/main.ts +++ b/frontend/app/src/main.ts @@ -187,8 +187,11 @@ function setReading(on: boolean): void { // score. Slowed down, because a human has to be able to follow and re-embody it. const TEMPOS = [0.5, 0.25, 1]; // performance opens at half speed; T cycles +const FULL_SPEED = TEMPOS.indexOf(1); let performing = false; -let tempoIdx = 0; +// Where T picks up from. The bench plays at full speed, so it starts here and the first +// press steps to 0.5x; performance mode re-seats it at 0.5x (index 0) when it opens. +let tempoIdx = FULL_SPEED; function setTempo(rate: number): void { renderer.setTempo(rate); @@ -205,11 +208,12 @@ function setPerforming(on: boolean): void { scoreTitleEl.textContent = on ? "the score · for the body" : "notation · the score"; if (on) { - tempoIdx = 0; + tempoIdx = 0; // performance opens at half speed — a body has to be able to follow it setTempo(TEMPOS[tempoIdx]); renderer.play(); } else { - setTempo(1); + tempoIdx = FULL_SPEED; + setTempo(TEMPOS[tempoIdx]); } }