Skip to content

feat(pi05): complete the native C++ runtime interface#144

Open
LiangSu8899 wants to merge 61 commits into
mainfrom
feat/pi05-cpp-runtime-interface
Open

feat(pi05): complete the native C++ runtime interface#144
LiangSu8899 wants to merge 61 commits into
mainfrom
feat/pi05-cpp-runtime-interface

Conversation

@LiangSu8899

@LiangSu8899 LiangSu8899 commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

  • add a native C++ PI0.5 producer that loads supported checkpoints through frt_model_runtime_open_v1
  • implement native prompt/state/image staging, tokenizer/formatter/embedding, fixed-window updates, graph capture, and logical action output
  • preserve the generic export/model-runtime ABI, explicit lifecycle, identity, stage, port, and capsule-region contracts
  • harden all C construction paths against STAGED declarations without matching verbs
  • derive native hardware identity from the active device and pin the shared producer face with golden canonical records

Dependency and merge order

This PR depends on #145 for the shared Python-free FA2 runtime. Merge #145 first, then rebase this branch before merging so the shared kernel change lands once through its owner review.

Design boundaries

  • FlashRT producer code owns checkpoint loading, preprocessing, model semantics, workspace, graph capture, and action postprocessing
  • consumers observe only declared ports, stages, regions, verbs, identity, and opaque handles
  • PI0.5 constants remain under model-owned directories; generic runtime, exec, modalities, and loader layers contain no PI0.5 semantics
  • Python, native CUDA, CPU/llama.cpp-style, and future producers use the same structural boundary without model_kind or backend_kind ABI fields
  • FlashRT has one exec implementation per linked process; heterogeneous instances enter through the capsule backend seam

Compatibility

The generic image face is strict RGB8 while the legacy vision entry preserves explicit RGB/BGR/RGBA/BGRA/GRAY conversion. Logical actions are F32; actions_raw is the BF16 SWAP alias. Port, stage, binding, or restore changes alter the fingerprint, so incompatible capsules must be regenerated.

Validation checklist

  • CUDA-off C++ build and CTest: 11/11
  • SM120 C++ build and CTest: 26/26
  • Python model-runtime contract: 29/29
  • Python runtime-export contract: 17/17
  • Python and C++ producers independently match the 8-record golden schema
  • native real-checkpoint open, observed-device identity, staging, output, and lifecycle
  • FP8 identical-input mechanism lane is bit-exact; logical actions satisfy the restored fixed allclose gate
  • full service-loop profiler range covers input staging, replay, and output readback
  • strict model image face and legacy image compatibility are both tested
  • public docs, migration notes, contribution guide, review checklist, and PR template updated
  • full diff contains no private paths, environment details, generated artifacts, or generic-layer model leakage

The official OpenPI E2E and Nsight kernel-sequence captures were accepted in the prior audit and were not recollected in this final documentation/contract pass.

@LiangSu8899

Copy link
Copy Markdown
Member Author

Shared FA2 ownership and packaging has been split into prerequisite PR #145. The model runtime continues to consume the same fvk_* C entries and does not carry a private kernel implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Track Jetson-PI multi-backend provider integration

1 participant