Adding TensorRT support could significantly speed up inference for NVIDIA GPUs. Should be an optional optimization step in the model loading pipeline.
Adding TensorRT support could significantly speed up inference for NVIDIA GPUs. Should be an optional optimization step in the model loading pipeline.