Skip to content

TensorRT

Install

Tip

To download tensorrt we must login to nvidia

  • download local repo package
  • install the package
  • register the gpg key
  • run apt update

  • install apt tensorrt

  • Download from nvidia

alt text

Install TensorRT
1
2
3
4
5
6
dpkg -i nv-tensorrt-local-repo-ubuntu2204-10.3.0-cuda-12.5_1.0-1_arm64.deb 
cp /var/nv-tensorrt-local-repo-ubuntu2204-10.3.0-cuda-12.5/nv-tensorrt-local-F9A70CFC-keyring.gpg /usr/share/keyrings/
apt update
apt install tensorrt
# check
dpkg -l | grep tensorrt

1
2
3
4
5
6
7
import tensorrt as trt

logger = trt.Logger(trt.Logger.WARNING)
runtime = trt.Runtime(logger)

with open("model.engine", "rb") as f:
    engine = runtime.deserialize_cuda_engine(f.read())

Reference

TensorRT In Docker