Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
3240324
Use autocxx bindings for NetworkDefinition, CudaEngine, EngineInspector
theHamsta Feb 18, 2026
793f4da
UniquePtr
theHamsta Feb 28, 2026
04b2b19
sa
theHamsta Feb 28, 2026
75f9dd0
Remove more locks
theHamsta Feb 28, 2026
4b3c48d
IProgressMonitor
theHamsta Feb 28, 2026
ea7788a
More UniquePtr
theHamsta Feb 28, 2026
7842b01
Almost remove mock
theHamsta Feb 28, 2026
79cded6
Remove mock
theHamsta Feb 28, 2026
42542b8
Pass mock test with as few mock as possible
theHamsta Feb 28, 2026
899d331
Fixup
theHamsta Feb 28, 2026
c3cd527
Move CudaEngine, EngineInspector to separate files
theHamsta Feb 28, 2026
eb00246
Fix docs
theHamsta Feb 28, 2026
677d8e2
Try to generated v_1_0 interfaces
theHamsta Feb 28, 2026
d93559d
Experiments with subclassing
theHamsta Mar 1, 2026
c178d51
Remove namespace real_bindings
theHamsta Mar 1, 2026
2cfbf9f
Remove real feature
theHamsta Mar 1, 2026
eeecd8b
Document ProgressMonitor
theHamsta Mar 1, 2026
98a9697
Clean up network a bit
theHamsta Mar 1, 2026
6d29e0a
Put interfaces in dedicated module
theHamsta Mar 1, 2026
e895ced
Add IErrorRecorder, IDebugListener
theHamsta Mar 1, 2026
6ff46b1
Generate ILogger
theHamsta Mar 1, 2026
87e1d12
SerializationConfig
theHamsta Mar 1, 2026
0a08db0
Remove empty test module
theHamsta Mar 1, 2026
6efb35e
Add OptimizationProfile
theHamsta Mar 1, 2026
8a7b5cd
Add OptimizationProfile
theHamsta Mar 1, 2026
7fe0557
Minimal mocking for basic_workflow
theHamsta Mar 2, 2026
8ca2ad6
Some more mocking
theHamsta Mar 2, 2026
b647a43
Unify define_network_layer/impl_layer_real
theHamsta Mar 4, 2026
fb5208f
Expose INormalizationLayer
theHamsta Mar 4, 2026
edf1490
Refactor Layer and Tensor
theHamsta Mar 5, 2026
1b3c857
Require network parameter for Layer/Tensor
theHamsta Mar 5, 2026
c5068ef
Implement Layer for indirect subclasses for ILayer
theHamsta Mar 5, 2026
0991bba
Add more layers
theHamsta Mar 5, 2026
ddfb01d
Mark TrtLayer unsafe
theHamsta Mar 5, 2026
41e2491
Add Attention, Loop, IfConditional
theHamsta Mar 5, 2026
76f5d0b
Extra network checks
theHamsta Mar 5, 2026
0e80755
Add strong type for Axes
theHamsta Mar 5, 2026
f7ecd2c
Make more setter bools a result
theHamsta Mar 5, 2026
30ed1cd
Remove real directory
theHamsta Mar 5, 2026
ca08e59
Add StreamReaderV2
theHamsta Mar 5, 2026
2a6d6cb
Add deserialize_cuda_engine_v2
theHamsta Mar 5, 2026
f0be9ec
Add Refitter
theHamsta Mar 5, 2026
e78114b
Draft for Refitter methods
theHamsta Mar 5, 2026
11ae915
Add refit test
theHamsta Mar 5, 2026
1cf53af
Test ProgressMonitor
theHamsta Mar 5, 2026
a93ba14
Improve lifetimes in network
theHamsta Mar 5, 2026
2545030
Fix interfaces
theHamsta Mar 5, 2026
d19c513
Update setting ErrorRecorder
theHamsta Mar 5, 2026
0c6676e
Accept Rc<RefCell<>> for interfaces
theHamsta Mar 5, 2026
734130d
Disable error_recorder
theHamsta Mar 5, 2026
58014f8
Add long lifetime requirements to weights
theHamsta Mar 6, 2026
f2f57b4
Prevent segfault
theHamsta Mar 6, 2026
334b6d9
Try to make Weights safer
theHamsta Mar 6, 2026
aa1359d
Use i64 dims where meaningful
theHamsta Mar 6, 2026
f525187
Type the return value of create_infer_builder
theHamsta Mar 7, 2026
9360e69
Implement ProgressMonitor more proper
theHamsta Mar 7, 2026
3313630
Fix bug in refitter
theHamsta Mar 7, 2026
35e8f3b
Bump bindgen
theHamsta Mar 7, 2026
4da5ddc
Bindgen workaround
theHamsta Mar 7, 2026
fee414e
Make interface methods use shared refs
theHamsta Mar 7, 2026
9a45ecb
Implement ErrorRecorder, GpuAllocator
theHamsta Mar 7, 2026
7d4a61b
Check for null in ProgressMonitor destructor
theHamsta Mar 7, 2026
7db86ac
Move interface to high level folder
theHamsta Mar 7, 2026
1a42284
Add size_bits on DataType
theHamsta Mar 7, 2026
e3e76a7
Only allow setting progress_monitor once
theHamsta Mar 7, 2026
c78cab6
Rename interface C functions
theHamsta Mar 7, 2026
af27960
Add note to progress_monitor comment
theHamsta Mar 7, 2026
9e3f2d2
Update docs
theHamsta Mar 7, 2026
644bc34
Make ProgressMonitor private
theHamsta Mar 7, 2026
439b0a9
Remove unnecessary get_unchecked_mut
theHamsta Mar 7, 2026
a9e3830
Rename HandleProgress to MonitorProgress
theHamsta Mar 7, 2026
62d62ce
Type allocator better
theHamsta Mar 7, 2026
33d1ee1
Fix ErrorRecorder
theHamsta Mar 7, 2026
af0b7c5
Allow setting ErrorRecorder on network and builder
theHamsta Mar 7, 2026
468b042
Use stronger types for trtx_* interface functions
theHamsta Mar 8, 2026
def3505
Add DebugListener
theHamsta Mar 8, 2026
bf40f3e
Test DebugListener
theHamsta Mar 8, 2026
f2f8b69
Improve lifetimes
theHamsta Mar 9, 2026
bed2675
Pin<&mut ITensor> -> *mut ITensor
theHamsta Mar 9, 2026
14f6c75
Don't require mut Tensor
theHamsta Mar 9, 2026
f1d05e9
Improve lifetimes, Create Axes::new()
theHamsta Mar 9, 2026
be64091
More i64
theHamsta Mar 9, 2026
8615b68
Fix lifetime for `set_input`, add TripLimit, Axes::from_slice
theHamsta Mar 11, 2026
b9502e6
Add another debug listener test
theHamsta Mar 11, 2026
2aca6b7
Fix clippy lints
theHamsta Mar 11, 2026
f5d80c0
Make DebugListener return result
theHamsta Mar 11, 2026
5f3d275
add_optimization_profile
theHamsta Mar 13, 2026
53fd0cb
Disable tests with reason
theHamsta Mar 13, 2026
a7a46c9
Add getLayer
theHamsta Mar 13, 2026
4e4de6c
Add get_layer test
theHamsta Mar 13, 2026
25380d1
Add EngineInspector tests
theHamsta Mar 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 1 addition & 76 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ By default, trtx builds with real TensorRT-RTX and cudarc for CUDA operations:

```bash
# Set environment
export TENSORRT_RTX_DIR=/path/to/tensorrt-rtx
export CUDA_ROOT=/usr/local/cuda
export LD_LIBRARY_PATH=/path/to/tensorrt-rtx/lib

# Build (uses real TensorRT-RTX)
make build # Debug build
Expand Down Expand Up @@ -69,29 +68,6 @@ make publish # Publish to crates.io

## Architecture

### Three-Layer FFI Design

```
┌─────────────────────────┐
│ Rust Safe API (trtx) │ <- RAII, Result<T, Error>, lifetimes
├─────────────────────────┤
│ Raw FFI (trtx-sys) │ <- Bindgen-generated from wrapper.hpp
├─────────────────────────┤
│ C Wrapper Layer │ <- wrapper.hpp/cpp (exception handling)
├─────────────────────────┤
│ TensorRT-RTX C++ API │ <- NVIDIA library
└─────────────────────────┘
```

**Why three layers?**
- TensorRT-RTX is C++ with exceptions and classes
- C wrapper provides `extern "C"` interface with opaque pointers
- C wrapper catches exceptions and converts to error codes
- Bindgen generates Rust FFI from C wrapper
- trtx crate provides safe Rust abstractions

See `docs/FFI_GUIDE.md` for detailed FFI development workflow.

### Two-Phase Workflow

**Build Phase (AOT):**
Expand All @@ -114,57 +90,6 @@ Runtime → Deserialize Engine → ExecutionContext → Bind Tensors → Execute
- `trtx/src/executor.rs`: High-level executor API (rustnn-compatible)
- `trtx/src/error.rs`: Error types

## FFI Bindings

### When Modifying FFI

1. **Update C wrapper** in `trtx-sys/wrapper.hpp` and `trtx-sys/wrapper.cpp`
2. **Rebuild** to regenerate bindings: `cargo clean -p trtx-sys && cargo build`
3. **Update mock** in `trtx-sys/build.rs` (`generate_mock_bindings`) and `trtx-sys/mock.c`
4. **Add safe wrapper** in appropriate `trtx/src/*.rs` file

### Naming Conventions

- C functions: `trtx_<class>_<method>` (e.g., `trtx_cuda_engine_get_tensor_name`)
- Types: `Trtx<ClassName>` (e.g., `TrtxCudaEngine`)
- Constants: `TRTX_<NAME>` (e.g., `TRTX_SUCCESS`)

### Error Handling Pattern

All FFI functions follow this signature:
```c
int32_t trtx_function_name(
// ... input parameters ...
char* error_msg, // Always second-to-last
size_t error_msg_len // Always last
);
```

Return `TRTX_SUCCESS` (0) on success, error code on failure.

## Mock Mode

Mock mode provides stub implementations for development without TensorRT-RTX:
- Development on machines without TensorRT-RTX (e.g., macOS)
- CI/CD on any platform
- API validation without GPU

**Use `--features mock`** when TensorRT-RTX is not available. When adding new FFI functions, update both real bindings AND mock implementations in `trtx-sys/build.rs` and `trtx-sys/mock.c`.

### Mock Implementation Files

- `trtx-sys/build.rs`: `generate_mock_bindings()` function defines Rust FFI stubs
- `trtx-sys/mock.c`: C implementations that return `TRTX_SUCCESS` with dummy data

## Important Notes

### Build System

- `trtx-sys/build.rs` uses bindgen to auto-generate `bindings.rs` from `wrapper.hpp`
- Generated file: `target/debug/build/trtx-sys-*/out/bindings.rs`
- Manual edits to generated files are **overwritten** on rebuild
- Changes must go in source files (`wrapper.hpp`, `wrapper.cpp`, `build.rs`)

### Memory Management

- **RAII everywhere**: Use `Drop` trait for automatic cleanup
Expand Down
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,8 @@ Mock mode provides stub implementations that allow you to:

The `trtx` crate has the following Cargo features:

- `default`: "real", "dlopen_tensorrt_onnxparser", "dlopen_tensorrt_rtx", "onnxparser", "v_1_3"
- `default`: "dlopen_tensorrt_onnxparser", "dlopen_tensorrt_rtx", "onnxparser", "v_1_3"
- `mock`: use this library in mock mode. TensorRT libraries and a Nvidia are no longer necessary for execution
- `real`: opposite of `mock` mode. TensorRT and Nvidia GPU are required for execution
- `dlopen_tensorrt_rtx`: enables dynamic loading of the TensorRT library via `trtx::dynamically_load_tensorrt`
- `dlopen_tensorrt_onnxparser`: enables dynamic loading of the TensorRT ONNX parser library via `trtx::dynamically_load_tensorrt_onnxparser`
- `links_tensorrt_rtx`: links the TensorRT library, `trtx::dynamically_load_tensorrt` is now optional
Expand Down
2 changes: 1 addition & 1 deletion docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ trtx-rs/
### 1. FFI Layer (trtx-sys)

**Approach:**
- Use bindgen to generate raw bindings
- Use autocxx to generate raw bindings
- Minimal manual intervention
- Direct mapping to C++ API
- No safety guarantees
Expand Down
Loading
Loading