Add multi-GPU (TensorRT Multi-Device) Triton inference example

[pkisfaludi-nv]

Summary

Adds an example showing a Morpheus pipeline running a single TensorRT engine sharded across multiple GPUs via the TensorRT Multi-Device (MD)–enabled Triton tensorrt backend — with no Morpheus code change.

Because TritonInferenceStage is a transparent gRPC client, MD (a single engine split across ≥2 GPUs by the Triton backend over NCCL, GA in TensorRT 11) is entirely server-side. The only difference between single-GPU and multi-GPU inference is model_name="mlp_sd" vs "mlp_md".

What's added (examples/multi_device_triton/)

• run.py — a LinearPipeline (InMemorySource → Deserialize → BuildInput → TritonInferenceStage → InMemorySink); --compare asserts the 2-GPU MD model matches the 1-GPU baseline.
• model_repo/mlp_md + model_repo/mlp_sd — batched Triton configs (the MD model adds only KIND_MODEL + enable_multi_device/multi_device_gpus/multi_device_per_rank_engines).
• build_tp_engines_batched.cpp — builds the TP-sharded MLP plans (column-parallel W1, row-parallel W2 + AllReduce, dynamic batch).
• triton_md_compare.py — a raw-gRPC correctness gate (validates the server independently of Morpheus).

Validation

On 8× B200 (NVLink), Morpheus 25.06-runtime, MD tensorrt backend on TensorRT 11.1 + NCCL, tritonserver:25.06-py3:

• Morpheus end-to-end run.py --compare: mlp_sd (1 GPU) vs mlp_md (2 GPU) → rel_max = 4.78e-3 PASS (pipeline identical except model_name).
• Raw-gRPC gate: rel_max = 3.56e-3, cos_min = 0.999999. Server logs TensorRT Multi-Device ready: 2 ranks; both GPUs hold the model.

Notes

• Requires a Triton server whose tensorrt backend is built with -DTRITON_ENABLE_TENSORRT_MULTI_DEVICE=ON (TensorRT ≥ 11, NCCL).
• run.py includes small, commented workarounds for a numba/cuda-python binding quirk in the 25.06-runtime container (cudf device→host repr; uses DLPack for the device handoff). They are cosmetic to data movement.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[pkisfaludi-nv]

@dagardner-nv (cc @nv-morpheus/morpheus-codeowners-eng) — this is now ready for review. It adds an examples/multi_device_triton/ example showing a Morpheus pipeline running multi-GPU TensorRT inference via an MD-enabled Triton backend with no Morpheus code change (only model_name differs); validated end-to-end on 8× B200 (rel_max 4.78e-3, mlp_sd vs mlp_md).

Two things I can't do from a fork and would appreciate help with:

• Label Checker is failing — it needs a category label (e.g. non-breaking/feature); could a maintainer add the appropriate one?
• copy-pr-bot notes CI won't run until a PR vetter approves — would appreciate a vet when convenient.

Happy to address any feedback. Thanks!

[willkill07]

/ok to test 0aa7534