Every few months a client sends us the same email: "We have one camera already installed. Can you get us distance from it?" Until recently the honest answer was mostly no — you got a pretty relative depth map, an arbitrary scale, and nothing you could put in a tolerance report. That answer has changed. Foundation depth models trained on tens of millions of images now produce dense, temporally reasonable depth from a single frame, and metric variants output something close to real metres.
"Close to" is doing a lot of work in that sentence. This tutorial shows how to run a monocular depth model inside an OpenCV 5 pipeline, how to convert its output into metres you can actually justify, how to measure the error honestly, and — the part most tutorials skip — how to decide whether you should be using one camera at all.
1. What monocular depth models actually give you
There are three different outputs sold under the word "depth", and confusing them is the most common failure we see in prototypes:
- Relative inverse depth (disparity-like). Depth Anything V2's standard checkpoints and MiDaS output this. Larger value = closer. There is no unit, and the scale and offset change from frame to frame. Useful for ordering, segmentation cues, bokeh, obstacle ranking — useless for measurement without alignment.
- Affine-invariant depth. Same thing with the promise that a single scale and shift per image maps it to true depth. That mapping still has to be estimated from something else.
- Metric depth. Checkpoints fine-tuned on metric datasets (Depth Anything V2 metric variants, UniDepth, Metric3D, ZoeDepth) output metres directly, conditioned implicitly or explicitly on camera intrinsics. Indoor and outdoor checkpoints are usually separate, and using the wrong one is a 2–3x error, not a 5% error.
Decide which one your business rule needs before you download anything. "Is the pallet closer than 2 m" is a metric question. "Which of these two people is nearer the door" is not.
2. Getting the model into ONNX
OpenCV 5's DNN module runs ONNX. Export from the PyTorch checkpoint rather than trusting a random ONNX file from a model zoo — you need to know the input size, the normalisation, and the output convention, and the only reliable way to know them is to export yourself.
import torch
from depth_anything_v2.dpt import DepthAnythingV2
CFG = dict(encoder="vits", features=64, out_channels=[48, 96, 192, 384])
model = DepthAnythingV2(**CFG)
model.load_state_dict(torch.load("depth_anything_v2_vits.pth", map_location="cpu"))
model.eval()
dummy = torch.randn(1, 3, 518, 518)
torch.onnx.export(
model, dummy, "dav2_vits_518.onnx",
input_names=["image"], output_names=["depth"],
opset_version=17,
dynamic_axes=None, # fixed shapes: faster and far fewer surprises
)
Two deliberate choices there:
- Fixed input shape. Dynamic axes look convenient and cost you graph optimisation, especially on TensorRT and OpenVINO. Pick one resolution, letterbox everything into it.
- Small encoder first. ViT-S at 518x518 is roughly 25 MB and runs in tens of milliseconds on a modern GPU. Prove the pipeline with it, then decide whether ViT-B or ViT-L buys you accuracy that matters. On most industrial scenes the jump from S to L changes the median error far less than fixing your intrinsics does.
Validate the export before writing any OpenCV code: run the same image through PyTorch and through onnxruntime, and check the max absolute difference is at the 1e-3 level. If it is not, your opset or normalisation is wrong and every downstream number is fiction.
3. Inference in OpenCV 5
The new DNN engine in OpenCV 5 handles transformer graphs considerably better than the 4.x one, but the pre-processing contract is still yours to get right. Depth Anything V2 expects RGB, [0,1], then ImageNet mean/std normalisation, with the side lengths a multiple of 14 (the ViT patch size).
import cv2
import numpy as np
INPUT = 518
MEAN = np.array([0.485, 0.456, 0.406], np.float32)
STD = np.array([0.229, 0.224, 0.225], np.float32)
net = cv2.dnn.readNetFromONNX("dav2_vits_518.onnx")
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA_FP16)
def preprocess(bgr):
rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
resized = cv2.resize(rgb, (INPUT, INPUT), interpolation=cv2.INTER_CUBIC)
normed = (resized - MEAN) / STD
return cv2.dnn.blobFromImage(normed, swapRB=False)
def infer_depth(bgr):
net.setInput(preprocess(bgr))
out = net.forward() # (1, H, W) or (1, 1, H, W)
d = np.squeeze(out)
return cv2.resize(d, (bgr.shape[1], bgr.shape[0]), interpolation=cv2.INTER_LINEAR)
Note the aspect ratio decision hidden in cv2.resize. Squashing a 16:9 frame into a square changes the apparent geometry and biases the model. For measurement work, letterbox with a constant border and crop the padding back off the output; for ranking work, the squash is usually acceptable. Whichever you choose, do the same thing at evaluation time as at run time — mismatched preprocessing between the two is the single most common source of "it was better in the notebook".
For visualisation, normalise per-clip rather than per-frame, or your colour map will flicker:
vis = np.clip((d - CLIP_LO) / (CLIP_HI - CLIP_LO), 0, 1)
vis = cv2.applyColorMap((vis * 255).astype(np.uint8), cv2.COLORMAP_INFERNO)
4. From relative output to metres
If you are running a relative checkpoint, you need a scale-and-shift alignment against some known reference. Least squares on inverse depth is the standard approach and takes four lines:
def align_scale_shift(pred_inv, gt_depth, mask):
"""Fit pred_inv ~ a * (1/gt) + b over valid pixels; return metric depth map."""
x = pred_inv[mask].reshape(-1, 1)
y = (1.0 / gt_depth[mask]).reshape(-1, 1)
A = np.hstack([x, np.ones_like(x)])
coef, *_ = np.linalg.lstsq(A, y, rcond=None)
a, b = coef.ravel()
inv_metric = a * pred_inv + b
return 1.0 / np.maximum(inv_metric, 1e-6)
Where does gt_depth come from on a real site? In practice, one of four places:
- A known plane. Mark three or more points on the floor or conveyor whose distance you measured with a tape or laser. Excellent for fixed cameras, which is most of industrial vision.
- A known object size. A pallet, a doorway, a licence plate, a person of known height. Cheap, noisy, and enough for coarse zoning.
- A one-off ranging sensor. A £30 ToF or laser module during commissioning, then removed.
- A homography you already have. If you calibrated the ground plane for tracking and counting, you already possess metric ground truth for every floor pixel — reuse it. Our multi-object tracking and counting tutorial sets that geometry up.
Metric checkpoints skip the fitting but not the sanity check. They are conditioned on assumed intrinsics; a wide-angle security lens on an indoor checkpoint trained mostly on phone photos will be confidently wrong. Always fit and record a residual scale factor even when you believe the model outputs metres. If that factor is 1.03 you are fine. If it is 1.6, the model is not measuring your scene, it is guessing it.
5. Undistort first, always
Every alignment above assumes a pinhole camera. Run the depth model on raw fisheye or heavily distorted frames and the error will be small in the centre and grotesque in the corners — exactly where your zone boundaries usually sit.
K, dist = load_intrinsics("cam03.yml")
newK, roi = cv2.getOptimalNewCameraMatrix(K, dist, (w, h), alpha=0)
map1, map2 = cv2.initUndistortRectifyMap(K, dist, None, newK, (w, h), cv2.CV_16SC2)
undist = cv2.remap(frame, map1, map2, cv2.INTER_LINEAR)
Calibrate the camera properly before you calibrate the depth. Our ChArUco calibration tutorial covers the capture protocol and how to read reprojection error without fooling yourself.
6. Temporal stability
Per-frame monocular depth flickers. Even video-tuned checkpoints drift in absolute scale across a clip. Two cheap fixes that cover most needs:
Exponential smoothing in inverse-depth space (linear in disparity, which is the space the model is smooth in):
inv = 1.0 / np.maximum(depth, 1e-3)
state = inv if state is None else (ALPHA * inv + (1 - ALPHA) * state)
depth_s = 1.0 / np.maximum(state, 1e-6)
Edge-aware refinement with a guided/joint bilateral filter, using the greyscale frame as guide, so smoothing does not bleed across object boundaries:
depth_s = cv2.ximgproc.jointBilateralFilter(
guide.astype(np.float32), depth_s.astype(np.float32), d=9,
sigmaColor=25, sigmaSpace=9)
For a moving camera, also lock the scale: fit the scale factor once against the ground plane, then hold it, rather than re-fitting each frame. Re-fitting per frame is how you get a system whose measurements breathe.
7. Measuring the error like an engineer
Do not ship a demo video. Ship a table. Collect 100–300 frames across the real operating envelope — near, far, bright, dark, empty, occluded — with ground truth from a laser distance meter, and report:
| Metric | What it tells you |
|---|---|
| AbsRel = mean( | d − d* |
| RMSE (m) | Penalises the far-field blowups |
| δ<1.25 | Fraction of pixels within 25%; catches catastrophic regions |
| Error vs. distance, binned | Where the model quietly gives up |
| Error vs. image region | Corners, edges, reflective floors |
Our rough field expectation for a well-aligned ViT-S/ViT-B pipeline on a fixed indoor camera: a few percent AbsRel out to roughly 5–8 m, degrading fast beyond that, with the worst errors on textureless walls, glass, dark clothing, and mirror-like floors. Treat those as your numbers to reproduce, not as a spec — the whole point is that the answer is scene-dependent.
Write the acceptance rule down before the trial: for example, "the system must classify presence in zone A/B/C correctly on 99% of frames, with distance error under 15 cm inside 4 m." A monocular depth system that cannot state its rule is a screensaver.
8. When one camera is the wrong answer
We say this to clients regularly, and it saves engagements:
- Tolerances tighter than a few percent, or sub-centimetre metrology. Use calibrated stereo or structured light. Monocular depth infers; stereo triangulates.
- Safety-rated stopping distance. Certification bodies want a deterministic sensor with a failure model. Use a lidar or a safety-rated ToF; the network can be an additional layer, not the primary one.
- Featureless or transparent scenes. Glass, mirrors, uniform white walls, dark liquids. Active sensing wins.
- Hard latency budgets on cheap edge hardware. A ViT at 518x518 on a Raspberry Pi-class NPU is not a 30 fps proposition; see our notes on choosing an embedded vision platform.
Where monocular depth genuinely shines: retrofits on existing single-camera installations, coarse zoning and safety-adjacent alerting, background separation and occlusion ordering for tracking, 3D-ish visualisation, auto-labelling depth for a smaller student model, and giving a detector context that a 2D box cannot ("that forklift is 12 m away, not 3 m").
9. A workable production shape
capture (GStreamer/RTSP) → undistort → letterbox
→ depth model (ONNX, FP16, fixed shape)
→ scale/shift alignment (fitted once, versioned)
→ temporal smoothing + guided filter
→ business logic on metric depth (zones, distances, ordering)
→ overlay + event stream
Keep the alignment coefficients in version control next to the camera intrinsics, tag them with the model hash, and re-verify both after any lens adjustment or camera bump. A depth pipeline whose calibration lives in someone's notebook is a pipeline that will silently drift out of spec.
If you are weighing single-camera depth against stereo or ToF for a real installation, that trade-off is a one- or two-week feasibility study, not a guess. Get in touch with your camera model, mounting geometry, working distance and the tolerance you actually need, and we will tell you which sensor — and which half of this pipeline — your project really requires.