diff --git a/docs/design/MOTION_CONTRACT.md b/docs/design/MOTION_CONTRACT.md index 27407da..f379a0e 100644 --- a/docs/design/MOTION_CONTRACT.md +++ b/docs/design/MOTION_CONTRACT.md @@ -30,6 +30,9 @@ > the PMX skeleton, rest pose and versioned humanoid mapping. > Revised again on 2026-09-21: `usd-motion-plugins` v0.5.0 shipped installable > `motionCore` and `motionRetarget`, and both adapters implemented §10 and §12. +> Revised 2026-09-25: MOT-O10 is narrowed by a measured comparison, MOT-O12 is +> opened, and §10.7 records that the shared core has promoted no common +> expression semantic yet. --- @@ -289,8 +292,9 @@ as a whole. | MOT-O2 | What a directly opened `.vmd` stage looks like. `usd-motion-plugins` fixes the frame (`/Animation`, the body as `UsdSkelAnimation`, `customData.motion`); what remains is that a VMD without a model has only control-rig tracks, which no evaluator can turn into body motion (§10.2) — so either the stage carries source tracks outside `Body`, or no such stage exists | `usd-motion-plugins`' `motionUsd` contract, then a consumer | | MOT-O4 | Camera and light tracks: any USD mapping at all | a consumer that needs one | | MOT-O8 | Whether `mmdControl` must also evaluate from a stage alone — `/Asset/rig` and `/Asset/morph` — for a runtime that holds no `CanonicalDocument` (§10.3) | a consumer that holds only the stage | -| MOT-O10 | The rest a clip from MMD states. Every MMD bone rests at identity rotation, so §12.3's rotations are relative to the model's modelled pose, in which every local character's upper arms point 37–42° below horizontal ([report](../reports/2026-09-19-phase9-roles-and-root.md)) — not the level arms a VRM's identity rest describes. Whether `mmdSkeletonAdapter` states a `SourceRestPose` measured from the rest bone directions (as the shared core's BVH profiles state `rest-offsets`), and against which reference directions, or leaves the difference to a retarget option | a measured A-pose-to-level-arm retarget comparison; the first generic skeletal acceptance used identity rest rotations and could not answer it ([report](../reports/2026-09-22-phase9-motion-acceptance.md)) | +| MOT-O10 | The rest a clip from MMD states. Every MMD bone rests at identity rotation, so §12.3's rotations are relative to the model's modelled pose, in which every local character's upper arms point 37–42° below horizontal — not the level arms a VRM's identity rest describes. Measured 2026-09-25 ([report](../reports/2026-09-25-phase9-rest-pose-comparison.md)): onto a level-arm skeleton, today's identity rest leaves every arm segment off by that angle (median 40°); a rest aimed from the rest bone directions of the **arm chain alone** (shoulder, upper arm, lower arm, hand) onto the lateral axis removes it (≤ 0.04°), while the same construction on the whole body flips the chest where `上半身3` lies below `上半身2` and levels feet no humanoid rest levels. A source rest is correct only with the same rest stated for a PMX **target**: stated on the source alone, PMX to PMX goes from a median 2.5° to 40°, and a level-arm source already reaches a PMX target 40° low today. So the source half waits for the target half | `usd-motion-plugins`: a retarget target rest distinct from the bind rest `/Asset/skel/Skeleton` states, and the T-pose directions as public vocabulary rather than `motionSource`'s private table | | MOT-O11 | Whether a plane link starts from its keyed rotation. §11.7 starts an enabled chain's plane angles at zero, so a knee's keyed rotation never reaches the pose and the knee is solved from straight; three.js r168 starts from the keyed rotation. On a motion that keys its legs' rotations alongside their goals, that start alone leaves a median 0.05 mm where §11.7 leaves 2.4 mm, and moves the knees a median 4.6 mm and at most 66 mm; on an IK-authored motion it changes little ([report](../reports/2026-09-19-phase9-ik-reference.md)). Which MMD does | MMD's output on a motion that keys its IK links | +| MOT-O12 | Where `上半身3` falls in the role table. In both local models that have it, the chain is `上半身` → `上半身3` → `上半身2` → `首`, with the neck and shoulders under `上半身2`; §12.2's version 1 maps `上半身2` to `chest` and `上半身3` to `upperChest`, against that order. A motion that keys `上半身2` then gives `neck` the chest's rotation a second time, cancelled only on a target that binds `upperChest`: onto targets without it, arms land a median 6–8° and at most 26° off ([report](../reports/2026-09-25-phase9-rest-pose-comparison.md)). Proposed: version 2 maps by the model's chain, `上半身3` to `chest` and `上半身2` to `upperChest` where `上半身3` is the ancestor | a table version 2, checked against models with `上半身3` in both orders if any exist | Resolved: @@ -453,7 +457,11 @@ the bound weight as a scalar. That source-preserving channel is retained even when an optional semantic expression is emitted beside it. The shared core owns common expression vocabulary; this repository owns any -mapping from an MMD source name to that vocabulary. Such a mapping is explicit, +mapping from an MMD source name to that vocabulary. As of `usd-motion-plugins` +v0.5.x there is none: a common semantic such as `face/blinkLeft` is promoted +only by a revision of the shared motion contract, never invented by a mapping +(its MOTION_CONTRACT.md §6), and none has been promoted, so this repository +emits source channels only. Such a mapping is explicit, versioned and limited to high-confidence conventions such as blink and basic mouth visemes. It preserves the original channel, diagnoses ambiguity and never turns an unknown model-specific morph into a guess. Generic motion code diff --git a/docs/reports/2026-09-19-phase9-roles-and-root.md b/docs/reports/2026-09-19-phase9-roles-and-root.md index e401cf4..b641bda 100644 --- a/docs/reports/2026-09-19-phase9-roles-and-root.md +++ b/docs/reports/2026-09-19-phase9-roles-and-root.md @@ -1,5 +1,7 @@ # Phase 9 humanoid roles and root motion against distributed models and motions (2026-09-19) +> Later: `上半身3`'s place in the version 1 table was found to run against the model's chain on 2026-09-25, MOT-O12 ([report](2026-09-25-phase9-rest-pose-comparison.md)). + Dated evidence from real runs; append-only ([contributing/documentation.md](../contributing/documentation.md)). diff --git a/docs/reports/2026-09-22-phase9-motion-acceptance.md b/docs/reports/2026-09-22-phase9-motion-acceptance.md index fb3b244..89f28f0 100644 --- a/docs/reports/2026-09-22-phase9-motion-acceptance.md +++ b/docs/reports/2026-09-22-phase9-motion-acceptance.md @@ -1,5 +1,7 @@ # Phase 9 skeletal motion acceptance (2026-09-22) +> Later: MOT-O10 was measured on 2026-09-25 ([report](2026-09-25-phase9-rest-pose-comparison.md)). + Dated evidence from real runs; append-only ([contributing/documentation.md](../contributing/documentation.md)). diff --git a/docs/reports/2026-09-25-phase9-rest-pose-comparison.md b/docs/reports/2026-09-25-phase9-rest-pose-comparison.md new file mode 100644 index 0000000..82864c7 --- /dev/null +++ b/docs/reports/2026-09-25-phase9-rest-pose-comparison.md @@ -0,0 +1,156 @@ +# Phase 9 A-pose rest against a level-arm rest (2026-09-25) + +Dated evidence from real runs; append-only +([contributing/documentation.md](../contributing/documentation.md)). + +## Question + +MOT-O10: every MMD bone rests at identity rotation, and in that rest a +character's arms hang 37–42° below horizontal +([roles report](2026-09-19-phase9-roles-and-root.md)). A VRM's identity rest +has level arms. Should `mmdSkeletonAdapter` state a `SourceRestPose` measured +from the rest bone directions, and if so, against which reference directions? + +## What was run + +Seventeen character models from the local corpus: every PMX over 1 MB that +resolves the required roles. Two pairs among them are variants of one +character each, and one 1.2 MB prop is skipped. Two dance +motions were used: A, 1.6 MB, sampled every 5th frame into 563 samples; and C, +11 MB, keys `上半身2`, sampled every 5th frame into 241 samples. All of these +are distributed files. None is a fixture, and none is committed or +redistributed +([DESIGN_POLICY.md §13](../design/DESIGN_POLICY.md#13-testing-policy)). The +runs were on Windows 11. No library of this repository was changed for them. + +A scratch program linked the installed `mmdControl`, `mmdSkeletonAdapter` and +`mmdMotionAdapter`, and the digest-pinned `motionRetarget` v0.5.0. For each +model it: + +1. bound each motion, evaluated it with `mmdControl`, and built the clip with + `mmdMotionAdapter::BuildClip`, as the adapters do today; +2. took the **ground truth** from MMD's own evaluation: the world direction + of 17 segments (spine, chest, neck, and each side's shoulder, upper arm, + lower arm, hand, upper leg, lower leg and foot). Each direction is the line + from one mapped joint's evaluated position to the next; +3. built the **stated rest** by the shared core's `t-pose` construction + (`motionSource`'s, copied into the scratch program only). Each role's world rest is the shortest rotation that turns + its bone onto the T-pose direction: arms lateral, legs down, spine up, feet + forward. The construction ran two ways: on the whole body, and on the arm + chain alone (shoulder, upper arm, lower arm, hand; fingers inherit); +4. retargeted with `PoseRetargeter` onto + - a **level-arm rig**: the model's own role joints, posed by those aims + into a T-pose, with identity rest rotations, as a normalized VRM + rests; + - **another PMX**: the next model in the list, as its stage states it, + with identity rests. It was also retargeted with the aims stated as that + target's rests, which `motionRetarget` accepts although the stage + does not state them; + - the second PMX, **from a level-arm source**: the clip the level-arm rig + received, carried on with identity source rests, as a VRMA clip would + arrive; +5. measured, per sample and segment, the angle between the retargeted + direction and the ground truth. + +Each figure below is the median over model pairs of each pair's median angle, +then the largest angle of any sample. Arm figures are the same for both +sides to within 0.4°. + +## Onto a level-arm rig + +The arm chain, motion A (C agrees to 0.01°): + +| Segment | Identity rests (today) | Stated arm-chain rest | +| --- | ---: | ---: | +| shoulder | 17.58° / 30.32° | 0.00° / 0.04° | +| upper arm | 40.16° / 42.42° | 0.00° / 0.04° | +| lower arm | 39.82° / 42.41° | 0.00° / 0.04° | +| hand | 39.74° / 42.38° | 0.00° / 0.04° | + +Today every arm segment is off by the model's own rest angle, the difference +MOT-O10 names, and it does not vary with the motion. Stating the arm chain's +rest removes it entirely. The torso and legs are 0.00° either way, because the +rig shares the model's geometry there. + +## The whole-body construction is not safe on MMD + +Applied to the whole body, the same construction fails in two places: + +- **The chest flips.** In both models that have `上半身3`, it lies + **below** `上半身2`: the line from `上半身2` to `上半身3` points 85° and 88° + downward. The construction aims the chest along that line, so it turns the + chest 175° and 178°. Retargeted onto another PMX, the arms and neck from those two + sources are then off by a median 100–170°, up to 179°. +- **The feet are levelled.** The rest line from `足首` to `足先EX` points + 40–66° below horizontal. Aiming it forward changes the rest by that much + (a median 48° over the pairs). A level-arm humanoid's foot slopes the same + way in its identity rest, so this change is not part of the A-pose + difference. + +The legs differ from straight down by a median 2.9° (upper) and 4.6° (lower), +at most 6.5°: small beside the arms. The line from `上半身2` to `首` leans +6–18° from vertical in every model. A level-arm humanoid straightens neither, +so the arm chain alone is where the A-pose differs. + +## Onto another PMX + +| Segment, motion A | Identity rests (today) | Source rest stated only | Stated on both sides | +| --- | ---: | ---: | ---: | +| shoulder | 7.51° / 22.28° | 17.58° / 30.32° | 0.00° / 0.04° | +| upper arm | 2.45° / 5.26° | 40.16° / 42.42° | 0.00° / 0.04° | +| lower arm | 1.82° / 5.62° | 39.82° / 42.41° | 0.00° / 0.03° | +| hand | 2.68° / 10.04° | 39.50° / 48.10° | 3.55° / 8.39° | + +| Segment, from a level-arm source, motion A | Identity target rests (today) | Target rest stated | +| --- | ---: | ---: | +| upper arm | 40.16° / 42.42° | 0.00° / 0.04° | +| lower arm | 39.82° / 42.41° | 0.00° / 0.04° | + +Today, PMX to PMX is off only by the difference between two models' rests. A +source rest stated **alone** makes it as wrong as a level-arm target is today. +The same 40° already reaches a PMX target from any source with a level-arm +rest. Stating the rest on **both** sides makes both directions exact. The +hand's remaining 3.55° is the hand-to-middle-finger line, which differs +between models' hands. It is 2.7–3.0° with identity rests. + +The stage cannot state a target's aim. `/Asset/skel/Skeleton`'s rests are +identity, and the adapter's descriptor must equal them +([report](2026-09-22-phase9-motion-acceptance.md)). The "stated" target above +exists only in the scratch program. Joints the program left unmapped kept +identity. A mapped joint the clip does not drive would take the aim as its +pose. + +## `上半身3` is out of the vocabulary's order + +In both models that have `上半身3`, the hierarchy is `上半身` → `上半身3` → +`上半身2` → `首`, with `首` and the shoulders' `肩P` under `上半身2`. +[MOTION_CONTRACT.md §12.2](../design/MOTION_CONTRACT.md#122-the-table-version-1) +maps `上半身2` to `chest` and `上半身3` to `upperChest`, so the vocabulary's +chest → upperChest runs against the model's own chain. For a motion that +keys `上半身2`, `neck` then carries the chest's rotation a second time. It is +cancelled only on a target that binds `upperChest` too. From those two +sources onto targets without `上半身3`, motion C leaves the arms a median 6–8° +off, 18–19° at the 95th percentile and 26° at most, with **today's** identity +rests. Motion A, which keys only `上半身`, is unaffected. This is independent +of MOT-O10. It is opened as MOT-O12. + +## Result and limits + +MOT-O10 is answered in part: + +- **What to state.** A rest aimed from the rest bone directions of the arm + chain alone, onto the lateral axis: the shared core's T-pose direction for + those four roles. It is exact on a level-arm rig. The same construction on + the whole body is wrong for MMD. +- **What it waits for.** The target half. A stated source rest is correct + only if a PMX target states the same rest. Without it, PMX to PMX gets + worse by the full rest angle. A target rest that differs from the bind rest + is not something `motionRetarget` v0.5.0 can take from a stage. Until it + can, the source half is not adopted. The T-pose directions are private to + `motionSource`, so adopting it here would also copy them (WORKSPACE.md + invariant 9). + +Not measured: roll about the bone. The shortest rotation leaves one, and +segment directions cannot see it. No clip was compared with a real VRM +model's rest, only with the level-arm rig built from each model's own +proportions. diff --git a/docs/roadmap/README.md b/docs/roadmap/README.md index bf705db..7941fbf 100644 --- a/docs/roadmap/README.md +++ b/docs/roadmap/README.md @@ -37,7 +37,7 @@ in.** No other document states a version for a Phase. | 6 | physics preservation | ✅ done | [v0.1.0](../releases/v0.1.0.md) | | 7 | VMD | ✅ done | [v0.1.0](../releases/v0.1.0.md) | | 8 | `MmdMaterialAPI`, renderer bridge and avatar runtime composition, including optional physics coupling | ⬜ | unassigned; schema/adapter work is here, composition is owned mostly outside this repository | -| 9 | shared motion core adoption — runs before Phase 8 | 🚧 skeletal evaluator, adapters and end-to-end acceptance done; expression interoperability and open rest-pose evidence remain | unassigned | +| 9 | shared motion core adoption — runs before Phase 8 | 🚧 skeletal evaluator, adapters and end-to-end acceptance done; the A-pose rest (MOT-O10) is measured and, with expression interoperability, waits on `usd-motion-plugins`; MOT-O12 remains | unassigned | Phases 0–7 ship together in v0.1.0, the first release, decided on 2026-09-17: it is the one that meets @@ -47,7 +47,7 @@ Phase 7 too. It is a 0.x release because no consumer has used the packages yet: Phase 8's consumer may still show a contract wrong ([current.md](current.md)). No earlier Phase had a release of its own. -Where things stand, as of 2026-09-22: +Where things stand, as of 2026-09-25: - The documentation baseline exists: the design policy, six focused design contracts, the workspace contract, and reference pages that state what is @@ -119,8 +119,12 @@ Where things stand, as of 2026-09-22: acceptance test now carries VMD-derived, IK-evaluated legs through digest-pinned `motionUsd` and `motionRetarget`, onto both a PMX-derived stage skeleton and a non-MMD skeleton - ([report](../reports/2026-09-22-phase9-motion-acceptance.md)). Expression - interoperability and MOT-O10 remain. + ([report](../reports/2026-09-22-phase9-motion-acceptance.md)). MOT-O10 was + measured on 2026-09-25: an arm-chain rest is exact, but only once a PMX + target can state it too, which waits on `usd-motion-plugins` + ([report](../reports/2026-09-25-phase9-rest-pose-comparison.md)). So does + expression interoperability: that repository has promoted no common + expression semantic. MOT-O12, found by the same run, remains. - Phase 8 follows Phase 9. Before renderer composition, this repository adds the admitted `MmdMaterialAPI`, applies it as a backward-compatible contract-v1 addition without renaming any `mmd:material:*` property, and @@ -151,7 +155,7 @@ schedules them. | MOT-O5 | Which MMD bones feed `RootMotion` | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | resolved in Phase 9 | | MOT-O6 | The humanoid role table and its version | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | resolved in Phase 9 | | MOT-O9 | MMD's own IK distance at a model's loop count | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | resolved in Phase 9 — §11.7 kept | -| MOT-O10 | The rest a clip from MMD states | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | measured A-pose-to-level-arm retarget comparison; the first generic acceptance used identity rest rotations | +| MOT-O10 | The rest a clip from MMD states | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | measured 2026-09-25; `usd-motion-plugins`: a target rest distinct from the bind rest, and public T-pose directions | | MOT-O2 | What a directly opened `.vmd` stage looks like | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | `usdVmdFileFormat` | | MOT-O3 | Which runtime owns MMD IK and append evaluation for baking | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | resolved in Phase 7; superseded 2026-09-17 — `mmdControl`, here | | PMX-O2 | QDEF verification | [PMX §16](../design/PMX_CONTRACT.md#16-open-questions) | a consumer | @@ -160,6 +164,7 @@ schedules them. | MOT-O4 | Camera and light tracks | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | a consumer | | MOT-O8 | Evaluating MMD motion from a stage alone | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | a consumer that holds only the stage | | MOT-O11 | Whether a knee starts from its keyed rotation | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | nothing (MMD's output to compare against) | +| MOT-O12 | Where `上半身3` falls in the role table | [MOTION §9](../design/MOTION_CONTRACT.md#9-open-questions) | a role-table version 2 | ## Quality bar (applies to every Phase) diff --git a/docs/roadmap/current.md b/docs/roadmap/current.md index 0bc5653..acfa6f1 100644 --- a/docs/roadmap/current.md +++ b/docs/roadmap/current.md @@ -1,7 +1,8 @@ # Phase 9, then Phase 8 — shared motion, material and avatar composition Status: 🚧 Phase 9 in progress — `mmdControl`, both shared-motion adapters, -skeletal end-to-end acceptance, MOT-O5, MOT-O6 and MOT-O9 are done; Phase 8 +skeletal end-to-end acceptance, MOT-O5, MOT-O6 and MOT-O9 are done; MOT-O10 +and expression interoperability wait on `usd-motion-plugins`; Phase 8 implementation has not started, but its `MmdMaterialAPI` and Hydra boundary are now decided. @@ -71,11 +72,22 @@ usd-avatar-runtime: composes the above per frame and coordinates rendering ([report](../reports/2026-09-19-phase9-roles-and-root.md)). No MMD bone is chosen as the root: it is the world transform of the joint `hips` maps to. MOT-O10, the rest a clip from MMD states, is opened with them. -- ⬜ **MOT-O10**: whether `mmdSkeletonAdapter` states a `SourceRestPose` - measured from the rest bone directions — MMD's arms rest in an A — decided - by a measured A-pose-to-level-arm retarget comparison. The first generic - skeletal acceptance used identity rest rotations and therefore could not - answer it. +- ⛔ **MOT-O10**: measured 2026-09-25 + ([report](../reports/2026-09-25-phase9-rest-pose-comparison.md)). Onto a + level-arm skeleton, today's identity rest leaves every arm segment a median + 40° low. A rest aimed from the arm chain's rest bone directions removes + that, and whole-body aiming is wrong for MMD. The source rest is correct + only if the same rest is stated for a PMX target: stated on the source + alone, PMX to PMX goes from 2.5° to 40°. So it waits for + `usd-motion-plugins` to take a target rest distinct from the stage's bind + rest, and to publish the T-pose directions. A level-arm clip already + reaches a PMX target 40° low today. +- ⬜ **MOT-O12**: `上半身3` sits between `上半身` and `上半身2` in both local + models that have it, against role-table version 1's `chest` → + `upperChest`. With a motion that keys `上半身2`, arms land up to 26° off on + targets without `upperChest`. A version-2 table that maps by the model's + chain is proposed + ([MOTION_CONTRACT.md §9](../design/MOTION_CONTRACT.md#9-open-questions)). - ✅ **`mmdMotionAdapter` and `mmdSkeletonAdapter`** (2026-09-21): digest-pinned `motionCore` and `motionRetarget` v0.5.0 packages; role-table version 1; stage-token `SkeletonDescriptor`, source rest and target `RetargetMap`; @@ -89,10 +101,12 @@ usd-avatar-runtime: composes the above per frame and coordinates rendering a separate local run retargeted a 1,201-sample dance onto a 539-joint PMX stage, passed `usdchecker`, and was observed moving in `usdview` ([report](../reports/2026-09-22-phase9-motion-acceptance.md)). -- ⬜ **Expression interoperability** follows the skeletal adapter path. Keep - every original `mmd:morph:` channel, then optionally emit only - explicit, versioned, high-confidence semantic mappings such as blink and - basic mouth visemes. Unknown model-specific morphs remain source channels +- ⛔ **Expression interoperability** waits for the shared core to promote a + common expression semantic. As of `usd-motion-plugins` v0.5.x none exists, + and one is promoted only by a revision of its motion contract, never by a + mapping. Until then, keep every original `mmd:morph:` channel. + When a semantic exists, optionally emit only explicit, versioned, + high-confidence mappings such as blink and basic mouth visemes. Unknown model-specific morphs remain source channels and generic motion code contains no MMD name table. Model visibility remains independently available under `mmd:model:visibility` ([MOTION_CONTRACT.md §10.7](../design/MOTION_CONTRACT.md#107-morphs-as-channels)).