Context
There's a rendering path (RendererBackend Protocol + PyrenderBackend, src/multicam_sim/dsl/render.py:100,110) and a depth path (DepthBackend, src/multicam_sim/dsl/depth.py:76), both open/closed Protocols. The pose layer emits ground-truth COCO-17 joints and, through the manifest, their per-camera 2D keypoints. What's missing is the loop that closes the evaluation: run a real 2D pose estimator on the rendered frames and compare its keypoints to ground truth.
What to do (larger)
Define a typed PoseEstimator Protocol (same open/closed style as RendererBackend): estimate(image) -> keypoints returning COCO-17 2D keypoints + confidences. Add an adapter for a real backend (RTMPose or ViTPose), gated behind an optional extra so the core stays import-light. Provide a helper that, for a scene, renders each camera/frame (via the existing RendererBackend), runs the estimator, and reports per-joint 2D error against the manifest's ground-truth uv (only where the joint is visible).
Acceptance criteria
PoseEstimator Protocol with a COCO-17 keypoint output type; the adapter implements it.- Heavy deps (mmpose/rtmpose or equivalent) are optional and guarded; tests skip cleanly when absent, and a fake/stub estimator exercises the compare-to-GT path on CPU.
- The comparison helper returns per-joint pixel error keyed by camera and frame, restricted to
visible joints.
ruff, mypy src, pytest -q green.
Pointers: dsl/render.py:100,107, dsl/depth.py:76, pose.py (COCO-17), manifest per_cam.uv. Renderer/pose design in DESIGN.md.
Context
There's a rendering path (
RendererBackendProtocol +PyrenderBackend,src/multicam_sim/dsl/render.py:100,110) and a depth path (DepthBackend,src/multicam_sim/dsl/depth.py:76), both open/closed Protocols. The pose layer emits ground-truth COCO-17 joints and, through the manifest, their per-camera 2D keypoints. What's missing is the loop that closes the evaluation: run a real 2D pose estimator on the rendered frames and compare its keypoints to ground truth.What to do (larger)
Define a typed
PoseEstimatorProtocol (same open/closed style asRendererBackend):estimate(image) -> keypointsreturning COCO-17 2D keypoints + confidences. Add an adapter for a real backend (RTMPose or ViTPose), gated behind an optional extra so the core stays import-light. Provide a helper that, for a scene, renders each camera/frame (via the existingRendererBackend), runs the estimator, and reports per-joint 2D error against the manifest's ground-truthuv(only where the joint isvisible).Acceptance criteria
PoseEstimatorProtocol with a COCO-17 keypoint output type; the adapter implements it.visiblejoints.ruff,mypy src,pytest -qgreen.Pointers:
dsl/render.py:100,107,dsl/depth.py:76,pose.py(COCO-17), manifestper_cam.uv. Renderer/pose design in DESIGN.md.