+1 (415) 360-7596
SentientSight

Real-time object detection on Jetson Orin Nano with OpenCV + TensorRT

The Jetson Orin Nano is the budget Jetson of 2026 and a very capable real-time detector when the pipeline is built correctly: camera frames captured through GStreamer, inference in TensorRT, and OpenCV for everything in between. This tutorial goes from a fresh JetPack install to a measured frames-per-second number, with the zero-copy and power tricks that make the difference between 12 FPS and 45.

1. JetPack setup

Flash the Orin Nano Developer Kit with the current JetPack for Orin (JetPack 6.2 or later; JetPack 7 is the current line and supports both Orin and Thor) using NVIDIA SDK Manager or the SD-card image. Then confirm what you have:

cat /etc/nv_tegra_release
dpkg -l | grep -E 'nvidia-jetpack|tensorrt|cuda-toolkit'
nvidia-smi 2>/dev/null || sudo tegrastats --interval 1000

tegrastats is the tool you will live in for the rest of this guide — it shows GPU load, memory and power rail readings in real time.

Install the build dependencies:

sudo apt update && sudo apt install -y build-essential cmake git pkg-config \
  libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
  libjpeg-dev libpng-dev libtiff-dev libv4l-dev python3-dev python3-numpy

2. Build OpenCV 5 with CUDA

The stock opencv-python wheel has no CUDA, and the apt package is old. Build OpenCV 5.0 from source against the JetPack CUDA toolkit. The Orin Nano's GPU is compute capability 8.7:

git clone -b 5.0.0 --depth 1 https://github.com/opencv/opencv.git
git clone -b 5.0.0 --depth 1 https://github.com/opencv/opencv_contrib.git
mkdir -p opencv/build && cd opencv/build
cmake -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_STANDARD=17 \
  -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
  -DWITH_CUDA=ON -DWITH_CUDNN=ON -DOPENCV_DNN_CUDA=ON \
  -DCUDA_ARCH_BIN=8.7 -DCUDA_ARCH_PTX= \
  -DWITH_GSTREAMER=ON -DWITH_V4L=ON \
  -DBUILD_opencv_python3=ON -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DBUILD_EXAMPLES=OFF \
  -DOPENCV_GENERATE_PKGCONFIG=ON ..
make -j$(nproc) && sudo make install && sudo ldconfig
python3 -c "import cv2; print(cv2.__version__); print(cv2.cuda.getCudaEnabledDeviceCount())"

Add swap before the build on the 8 GB board; the link step is memory-hungry. Expect an hour or two. C++17 is mandatory in OpenCV 5, which the JetPack GCC handles fine.

3. Camera capture through GStreamer

For a CSI camera (IMX219/IMX477) use nvarguscamerasrc; for USB cameras use v4l2src. Either way, let the hardware do the color conversion and keep OpenCV out of the decode path:

import cv2

def csi_pipeline(w=1280, h=720, fps=30):
    return (
        f"nvarguscamerasrc sensor-id=0 ! "
        f"video/x-raw(memory:NVMM), width={w}, height={h}, framerate={fps}/1, format=NV12 ! "
        f"nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! "
        f"appsink drop=true max-buffers=2"
    )

cap = cv2.VideoCapture(csi_pipeline(), cv2.CAP_GSTREAMER)
assert cap.isOpened(), "check the GStreamer build flag and the camera"

drop=true max-buffers=2 is important: if inference is slower than the camera, you want the newest frame, not a growing backlog. Note that in OpenCV 5 cap.get() returns -1 for properties a backend does not support, so test with < 0.

4. Build the TensorRT engine

Export YOLO26 to ONNX on any machine (see Running YOLO26 in OpenCV 5), then build the engine on the Jetson — TensorRT engines are specific to the GPU and TensorRT version:

/usr/src/tensorrt/bin/trtexec --onnx=yolo26n.onnx --saveEngine=yolo26n_fp16.engine \
  --fp16 --memPoolSize=workspace:2048 --useCudaGraph

FP16 is the sweet spot on Orin Nano. INT8 roughly doubles throughput again but needs a calibration set; build it with --int8 --calib=<cache> once you have validated accuracy on your own data. Cache the engine per JetPack version and rebuild it in CI when JetPack changes.

5. Inference loop with zero-copy

The naive loop copies the frame host to device, runs, copies back. On a shared-memory SoC like Orin the copies are pure waste. Use pinned (page-locked) host memory and a CUDA stream so capture, pre-processing and inference overlap:

import numpy as np
import tensorrt as trt
import pycuda.autoinit
import pycuda.driver as cuda

logger = trt.Logger(trt.Logger.WARNING)
with open("yolo26n_fp16.engine", "rb") as f, trt.Runtime(logger) as rt:
    engine = rt.deserialize_cuda_engine(f.read())
ctx = engine.create_execution_context()
stream = cuda.Stream()

in_name, out_name = engine.get_tensor_name(0), engine.get_tensor_name(1)
in_shape = tuple(engine.get_tensor_shape(in_name))     # (1, 3, 640, 640)
out_shape = tuple(engine.get_tensor_shape(out_name))   # (1, 300, 6)

h_in = cuda.pagelocked_empty(int(np.prod(in_shape)), np.float16)
h_out = cuda.pagelocked_empty(int(np.prod(out_shape)), np.float32)
d_in = cuda.mem_alloc(h_in.nbytes)
d_out = cuda.mem_alloc(h_out.nbytes)
ctx.set_tensor_address(in_name, int(d_in))
ctx.set_tensor_address(out_name, int(d_out))

def preprocess(frame):
    lb = cv2.resize(frame, (640, 640))                    # letterbox in production
    blob = cv2.dnn.blobFromImage(lb, 1 / 255.0, (640, 640), swapRB=True)
    np.copyto(h_in, blob.ravel().astype(np.float16))

while True:
    ok, frame = cap.read()
    if not ok:
        break
    preprocess(frame)
    cuda.memcpy_htod_async(d_in, h_in, stream)
    ctx.execute_async_v3(stream_handle=stream.handle)
    cuda.memcpy_dtoh_async(h_out, d_out, stream)
    stream.synchronize()
    dets = h_out.reshape(out_shape)[0]
    dets = dets[dets[:, 4] > 0.25]
    for x1, y1, x2, y2, s, c in dets:
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)

Two further steps remove the remaining copies: do the resize and normalization on the GPU with cv2.cuda.resize on a cv2.cuda_GpuMat and hand its device pointer to TensorRT, and use NVMM buffers end to end with the nvdsinfer/DeepStream path when you outgrow a hand-rolled loop. For a single camera, the pinned-memory version above usually already gets you out of the copy-bound regime.

Confirm the output tensor shape from the engine rather than assuming; YOLO26's end-to-end export gives (1, 300, 6), an older model gives (1, 84, 8400) and needs NMS.

6. Power and performance tuning

sudo nvpmodel -q                 # current power mode
sudo nvpmodel -m 0               # MAXN (highest; check the mode list for your module)
sudo jetson_clocks                # lock clocks at max for benchmarking
sudo tegrastats --interval 500    # watch GPU %, EMC %, power rails

Benchmark with clocks locked so you measure the pipeline and not the governor. Then pick the lowest power mode that still holds your target FPS: on a thermally constrained enclosure, a 15 W mode running steadily beats a 25 W mode that throttles after four minutes. Watch the EMC (memory controller) figure in tegrastats: if it is pegged while the GPU is idle, you are copy-bound and step 5 is where the time is.

Typical results for YOLO26n at 640x640 in FP16 on Orin Nano are comfortably real-time for a single 30 FPS camera; INT8 and a smaller input (512 or 416) buy headroom for a second camera or a heavier model. Record your own numbers per power mode and keep them with the engine cache.

7. Ship it

  • Pin the JetPack, TensorRT and OpenCV versions in a container and rebuild engines in CI when any of them changes.
  • Keep the camera pipeline string in config, not code; USB-to-CSI swaps should not need a rebuild.
  • Log tegrastats summaries in production; thermal throttling is the most common "it worked on the bench" failure.
  • Plan the module lifecycle: NVIDIA pulled forward end-of-life on several LPDDR4 Jetson modules in 2026, so check last-time-buy dates for anything you design in. Our edge AI page covers the planning side.

Need a production-grade Jetson pipeline built or tuned? Contact us.