Port to ort 2.0.0-rc.12 (ndarray 0.17) - #3
seanianallen wants to merge 1 commit into
Conversation
- Session::run now requires &mut Session: wrap the session in a Mutex, preserving the crate's &self inference API; the lock is held per inference call - SessionOutputs lost its second lifetime parameter - builder option methods return Error<SessionBuilder> (not Send+Sync): strip the recovery payload before boxing - ModelMetadata getters return Option; Session/Outlet fields are now behind accessors - SessionOutputs borrows only the output-name strings from the session; a documented transmute widens that borrow past the mutex guard
|
Heads-up: ort 2.0.0-rc.13 shipped this week (ONNX Runtime 1.28 baseline). I've validated this port against rc.13 as well — the only deltas are EP compile-time feature-gating and custom-operator API changes, neither of which orp touches, so this branch bumps cleanly. Happy to update the PR to rc.13 directly if you'd prefer to land on the latest rc, or keep it at rc.12 as submitted — whichever is easier to review. An rc.13 branch is ready at seanianallen/orp:ort-rc13 either way. |
|
Hi @seanianallen, sorry for the delay and thank you very much for this very useful contribution. I think that, while we're at it, and given that rc.13 is considered production ready, it makes sense to move directly to that version. Note that I will be shortly releasing the current state of orp and related crates so we can start on a clean state, and let people use them if they prefer to keep using rc9. |
Hi! This ports orp to
ort 2.0.0-rc.12/ndarray 0.17, keeping the public API unchanged. Companion PR for gline-rs is open as well.Why
ort 2.0.0-rc.9targets ONNX Runtime 1.20.x, which is increasingly hard to pair with current system installs (Homebrew and most distros now ship 1.24+). rc.12 supports ORT 1.17–1.24 via multiversioning, and empirically works against 1.26 withload-dynamic.What changed (all driven by rc.9 → rc.12 API changes)
Session::runnow requires&mut Session. To preserve this crate's&selfinference API (and theComposabletrait contracts),Modelwraps the session in aMutexheld per inference call — ONNX Runtime parallelizes internally via intra-op threads, so concurrent callers serialize at the run boundary only.SessionOutputslost its second lifetime parameter. It now only borrows the output name strings from the session's outlet metadata (values are ownedDynValues); a documentedtransmutewidens that borrow from the mutex-guard region to the model borrow'a, keeping thePostProcessor<'a, …>trait signatures unchanged.Error<SessionBuilder>, whose recovery payload is neitherSendnorSync; a small helper strips the payload so?conversion toBox<dyn Error + Send + Sync>keeps working.ModelMetadatagetters returnOption;Session/Outletfields moved behind accessors (inputs(),outputs(),name()",dtype()`).Builds clean with
--features load-dynamic; exercised end-to-end via the gline-rs test suite (17/17) against ONNX Runtime 1.26.Happy to adjust anything to your taste — in particular the Mutex vs. exposing
&mutinference, if you'd rather break API.