Skip to content

Port to ort 2.0.0-rc.12 (ndarray 0.17) - #3

Open
seanianallen wants to merge 1 commit into
fbilhaut:mainfrom
seanianallen:ort-rc12
Open

seanianallen wants to merge 1 commit into
fbilhaut:mainfrom
seanianallen:ort-rc12

Conversation

@seanianallen

Copy link
Copy Markdown

Hi! This ports orp to ort 2.0.0-rc.12 / ndarray 0.17, keeping the public API unchanged. Companion PR for gline-rs is open as well.

Why

ort 2.0.0-rc.9 targets ONNX Runtime 1.20.x, which is increasingly hard to pair with current system installs (Homebrew and most distros now ship 1.24+). rc.12 supports ORT 1.17–1.24 via multiversioning, and empirically works against 1.26 with load-dynamic.

What changed (all driven by rc.9 → rc.12 API changes)

  • Session::run now requires &mut Session. To preserve this crate's &self inference API (and the Composable trait contracts), Model wraps the session in a Mutex held per inference call — ONNX Runtime parallelizes internally via intra-op threads, so concurrent callers serialize at the run boundary only.
  • SessionOutputs lost its second lifetime parameter. It now only borrows the output name strings from the session's outlet metadata (values are owned DynValues); a documented transmute widens that borrow from the mutex-guard region to the model borrow 'a, keeping the PostProcessor<'a, …> trait signatures unchanged.
  • Builder option methods return Error<SessionBuilder>, whose recovery payload is neither Send nor Sync; a small helper strips the payload so ? conversion to Box<dyn Error + Send + Sync> keeps working.
  • ModelMetadata getters return Option; Session/Outlet fields moved behind accessors (inputs(), outputs(), name()", dtype()`).

Builds clean with --features load-dynamic; exercised end-to-end via the gline-rs test suite (17/17) against ONNX Runtime 1.26.

Happy to adjust anything to your taste — in particular the Mutex vs. exposing &mut inference, if you'd rather break API.

- Session::run now requires &mut Session: wrap the session in a Mutex,
  preserving the crate's &self inference API; the lock is held per
  inference call
- SessionOutputs lost its second lifetime parameter
- builder option methods return Error<SessionBuilder> (not Send+Sync):
  strip the recovery payload before boxing
- ModelMetadata getters return Option; Session/Outlet fields are now
  behind accessors
- SessionOutputs borrows only the output-name strings from the session;
  a documented transmute widens that borrow past the mutex guard
@seanianallen

Copy link
Copy Markdown
Author

Heads-up: ort 2.0.0-rc.13 shipped this week (ONNX Runtime 1.28 baseline). I've validated this port against rc.13 as well — the only deltas are EP compile-time feature-gating and custom-operator API changes, neither of which orp touches, so this branch bumps cleanly. Happy to update the PR to rc.13 directly if you'd prefer to land on the latest rc, or keep it at rc.12 as submitted — whichever is easier to review. An rc.13 branch is ready at seanianallen/orp:ort-rc13 either way.

@fbilhaut

fbilhaut commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Hi @seanianallen, sorry for the delay and thank you very much for this very useful contribution.

I think that, while we're at it, and given that rc.13 is considered production ready, it makes sense to move directly to that version.

Note that I will be shortly releasing the current state of orp and related crates so we can start on a clean state, and let people use them if they prefer to keep using rc9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants