Most vision projects we are asked to rescue arrive framed as a model problem. Accuracy is 91% and the client needs 99%, so the brief is "retrain it, or try a bigger network". Then we look at the frames. Half are motion-blurred, the exposure swings two stops between the morning and afternoon shift, the part is backlit by a window that did not exist during the pilot, and a single overhead LED has aged enough to shift the white point.
No architecture fixes that. Image quality is the cheapest accuracy you will ever buy, and it is almost always the part nobody owns. This tutorial is the acquisition side of an OpenCV 5 pipeline: how to control the sensor, how to handle high dynamic range and low light, how to correct vignetting and colour, and how to build an input-quality gate so bad frames are rejected at ingest rather than silently degrading your metrics.
1. Fix the optics and lighting before you write code
The order of operations matters, because every step below is damage limitation for something you could have prevented:
- Lighting. Controlled, diffuse, and constant. For inspection, dome or bar lights with a polariser beat ambient light every time. A dark enclosure plus your own illumination beats a bright room, because a room has weather and shifts.
- Exposure and gain. Short exposure kills motion blur; low gain kills noise. You can only have both if you have enough light — which is why lighting comes first.
- Lens and aperture. Stop down a little for depth of field and to push distortion and vignetting toward the middle of their range.
- Then software.
A number worth putting in front of every client: motion blur in pixels is roughly v * t / GSD, where v is part speed, t is exposure time and GSD is your ground sample distance (metres per pixel). A part moving at 0.5 m/s, a 5 ms exposure and a GSD of 0.2 mm/px gives 0.5 × 0.005 / 0.0002 = 12.5 px of smear. If your defect is 6 px across, the project is already over. Either shorten exposure to 1 ms and add light, or switch to a strobe synchronised to a trigger. This is a lighting-budget conversation, not a model-tuning conversation.
2. Take manual control of the camera
Auto-exposure, auto-white-balance and autofocus are the enemies of reproducibility. They mean the same scene produces different pixels on different days, which makes your thresholds meaningless and your calibration (see our ChArUco and stereo calibration walkthrough) drift.
With a UVC camera through the OpenCV VideoCapture API:
import cv2
cap = cv2.VideoCapture(0, cv2.CAP_V4L2)
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*"MJPG"))
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
# 1 = manual on most V4L2 UVC drivers, 3 = aperture priority / auto
cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 1)
cap.set(cv2.CAP_PROP_EXPOSURE, 156) # driver units, NOT milliseconds
cap.set(cv2.CAP_PROP_GAIN, 0)
cap.set(cv2.CAP_PROP_AUTOFOCUS, 0)
cap.set(cv2.CAP_PROP_FOCUS, 30)
cap.set(cv2.CAP_PROP_AUTO_WB, 0)
cap.set(cv2.CAP_PROP_WB_TEMPERATURE, 4600)
for prop, name in [(cv2.CAP_PROP_EXPOSURE, "exposure"),
(cv2.CAP_PROP_GAIN, "gain"),
(cv2.CAP_PROP_AUTO_WB, "auto_wb")]:
print(name, cap.get(prop)) # read back — many sets silently fail
Three hard-won rules:
- Always read back what you set.
VideoCapture.set()returnsTrueon plenty of drivers that then ignore the value. The read-back is your only proof. - Units are driver-defined.
CAP_PROP_EXPOSUREmay be in 100 µs ticks, arbitrary steps, or a negative log scale depending on the backend. Calibrate the mapping once with a light meter or a known scene and write the table into your repo. - For any serious industrial job, bypass
VideoCapture. GenICam/GigE Vision cameras should be driven by the vendor SDK (or Aravis, or Spinnaker) and handed to OpenCV as a buffer. You get real trigger control, exact exposure in microseconds, and no MJPEG recompression. OpenCV is your processing library, not your camera driver.
Then discard the first frames. Sensors need a moment after a settings change, and rolling-shutter cameras in particular return one or two stale frames:
for _ in range(5):
cap.read()
3. Flat-field correction: kill vignetting and dust
Every lens is darker at the edges, and every real installation eventually gets a dust speck on the sensor or a fingerprint on the window. Flat-field correction removes both, and it takes two calibration captures: a dark frame (lens capped, same exposure) and a flat frame (uniform white target filling the frame, well exposed but not clipped).
import numpy as np
def average_frames(cap, n=64):
acc = None
for _ in range(n):
ok, f = cap.read()
if not ok:
continue
f = f.astype(np.float32)
acc = f if acc is None else acc + f
return acc / n
dark = average_frames(cap) # lens capped
flat = average_frames(cap) # uniform white target
gain_map = (flat - dark)
gain_map = np.mean(gain_map) / np.clip(gain_map, 1e-3, None)
np.save("gain_map.npy", gain_map)
np.save("dark.npy", dark)
def flat_field(frame, dark, gain_map):
out = (frame.astype(np.float32) - dark) * gain_map
return np.clip(out, 0, 255).astype(np.uint8)
Averaging 64 frames matters: a single dark or flat frame carries its own noise, which you would then bake permanently into every image. Recapture the flat field whenever the lens, aperture or lighting changes, and store it next to the camera calibration file with the same serial numbers and timestamps. After correction, a uniform target should be flat to within a couple of percent corner-to-centre; measure it and record the number, because a rising corner falloff over months is how you detect a failing light.
4. High dynamic range: exposure fusion beats tone mapping for CV
Shiny metal under a bright light, or an outdoor camera looking at a shaded loading bay next to sunlit tarmac, exceeds what 8 bits can hold. Two or three frames at different exposures usually solve it.
For computer vision, prefer Mertens exposure fusion. It needs no exposure-time metadata, produces a well-behaved 8-bit output directly, and does not require you to calibrate a camera response function:
import cv2
shots = [cv2.imread("e_short.png"), cv2.imread("e_mid.png"), cv2.imread("e_long.png")]
# Align first — hand-held or vibrating rigs shift between exposures
aligner = cv2.createAlignMTB()
aligner.process(shots, shots)
merge = cv2.createMergeMertens(contrast_weight=1.0,
saturation_weight=1.0,
exposure_weight=0.0)
fused = merge.process(shots) # float32, 0..1
fused8 = np.clip(fused * 255, 0, 255).astype(np.uint8)
Set exposure_weight=0.0 when you care about detail rather than a photographically pleasing image — the well-exposedness term pulls mid-grey toward the centre and can flatten exactly the contrast you are trying to detect.
If you genuinely need a radiometric HDR (measuring relative luminance, not just seeing detail), use the Debevec path with real exposure times and then tone-map:
times = np.array([1/500., 1/125., 1/30.], dtype=np.float32)
cal = cv2.createCalibrateDebevec()
response = cal.process(shots, times)
hdr = cv2.createMergeDebevec().process(shots, times, response)
ldr = cv2.createTonemapDrago(gamma=1.8, saturation=0.9).process(hdr)
Two warnings. First, tone mapping is non-linear and spatially varying — never measure intensity or do photometric thresholding on a tone-mapped image. Keep the linear HDR for measurement and use the tone-mapped version only for display, or for feeding a network trained on ordinary photographs. Second, HDR bracketing costs frame rate and breaks on fast motion. On a moving line the correct fix is a wider-dynamic-range sensor or better lighting, not three exposures.
5. Low light: denoise without destroying the signal
When you cannot add light, you are trading noise against detail. Order the options by cost.
Temporal averaging is free and nearly lossless — if the scene is static. Averaging N frames cuts noise by the square root of N:
acc = np.zeros(shape, np.float32)
for _ in range(8):
ok, f = cap.read()
cv2.accumulate(f.astype(np.float32), acc)
acc /= 8
For scenes with motion, a running average with motion masking works, or move to cv2.fastNlMeansDenoisingColoredMulti, which uses a small stack of consecutive frames:
frames = [f0, f1, f2, f3, f4]
out = cv2.fastNlMeansDenoisingColoredMulti(
frames, imgToDenoiseIndex=2, temporalWindowSize=5, h=6, hColor=6,
templateWindowSize=7, searchWindowSize=21)
Spatial-only options, cheapest first: cv2.GaussianBlur (destroys edges — only for heavy pre-smoothing before a downscale), cv2.bilateralFilter (edge-preserving, slow at large d), cv2.fastNlMeansDenoising (best classical quality, slowest), and learned denoisers exported to ONNX and run through the DNN module if the frame budget allows.
Then two rules that matter more than the filter choice:
- Denoise before, not after, contrast enhancement. CLAHE amplifies noise enthusiastically. Denoise then CLAHE, never the reverse.
- If you feed a neural network, denoise conservatively or not at all. Heavy NLM removes exactly the fine texture a defect detector keys on. Match inference-time preprocessing to what the model was trained on, and if you intend to denoise in production, denoise the training set the same way. Mismatched preprocessing is one of the most common causes of the "worked in the notebook, failed on the line" pattern.
For contrast, use CLAHE on the luminance channel only — never on BGR channels independently, which shifts colours:
lab = cv2.cvtColor(frame, cv2.COLOR_BGR2LAB)
l, a, b = cv2.split(lab)
l = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8)).apply(l)
frame_eq = cv2.cvtColor(cv2.merge((l, a, b)), cv2.COLOR_LAB2BGR)
Keep clipLimit modest (1.5–3.0). Above that you are manufacturing texture, and downstream thresholds will chase it.
6. Colour: white balance and a chart, if colour is a decision variable
If your pipeline classifies by colour — ripeness, wire insulation, paint defects, product variants — then "the camera looks about right" is not good enough, because LEDs shift as they age and as they warm up.
Cheap and surprisingly effective, with a neutral surface in frame:
wb = cv2.xphoto.createSimpleWB()
balanced = wb.balanceWhite(frame)
grey = cv2.xphoto.createGrayworldWB()
grey.setSaturationThreshold(0.95)
balanced = grey.balanceWhite(frame)
If colour accuracy is contractual, put a ColorChecker in the scene during commissioning and fit a colour correction matrix with the mcc module (cv2.mcc.CCheckerDetector, then ColorCorrectionModel), storing the resulting 3×3 or polynomial transform alongside the flat field and the intrinsics. Re-measure on a schedule. One deployment we audited had drifted enough over fourteen months of LED ageing that a colour-based accept/reject rule had quietly inverted on one product line — the code never changed, the light did.
7. The input-quality gate
Here is the part that turns all of the above into something operational. Score every frame on cheap metrics, reject or flag the bad ones, and export the scores as telemetry. A vision system that says "frame rejected: too dark" is debuggable at 2 a.m.; one that just returns a wrong answer is not.
import cv2, numpy as np
def frame_quality(bgr):
g = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)
return {
# Focus/blur: variance of Laplacian. The threshold is scene-specific —
# calibrate it on your own good and bad frames, never copy a number.
"focus": float(cv2.Laplacian(g, cv2.CV_64F).var()),
"mean": float(g.mean()),
"rms_contrast": float(g.std()),
"clipped_high": float((g >= 254).mean()),
"clipped_low": float((g <= 1).mean()),
# Cheap noise proxy: high-frequency residual after a small median
"noise": float(cv2.absdiff(g, cv2.medianBlur(g, 3)).mean()),
}
LIMITS = {"focus": (120.0, None), "mean": (55.0, 200.0),
"rms_contrast": (18.0, None), "clipped_high": (None, 0.02),
"clipped_low": (None, 0.05), "noise": (None, 4.0)}
def gate(q):
reasons = []
for k, (lo, hi) in LIMITS.items():
if lo is not None and q[k] < lo:
reasons.append(f"{k}_low({q[k]:.1f}<{lo})")
if hi is not None and q[k] > hi:
reasons.append(f"{k}_high({q[k]:.2f}>{hi})")
return (len(reasons) == 0), reasons
How to use it in production:
- Derive the limits empirically. Collect a few hundred frames you and the operator agree are good, plus a deliberately bad set (defocused, dark, over-lit, blurred), and set thresholds where the distributions separate. Percentile-based limits from the good set — say the 1st percentile of
focus— are a reasonable starting point. - Two tiers, not one. Reject frames that cannot yield a valid result (retrigger, or raise an alarm). Flag frames that are merely marginal, keep their result, and mark it low-confidence.
- Log every score, always. Push mean brightness and focus to your metrics stack per camera. Slow drift in those series is the earliest possible warning of a dying lamp, a loosening lens, a dirty window or a thermal problem — long before accuracy visibly drops. That pairs directly with the golden-frame and drift alarms in our regression testing tutorial.
- Keep a rolling ring buffer of rejected frames on disk. When the client says "it failed at 14:20", you want the pixels.
Note the ordering discipline: quality scoring runs on the corrected frame (flat-fielded, white-balanced) but before any enhancement, denoise or resize. Score after CLAHE and every frame looks contrast-rich, including the broken ones.
8. A complete ingest stage
Wiring it together, this is what the front of a defensible pipeline looks like:
class Ingest:
def __init__(self, cap, dark, gain_map, K=None, dist=None, size=None):
self.cap, self.dark, self.gain_map = cap, dark, gain_map
self.maps = None
if K is not None:
self.maps = cv2.initUndistortRectifyMap(
K, dist, None, K, size, cv2.CV_16SC2)
def read(self):
ok, raw = self.cap.read()
if not ok:
return None, ["capture_failed"], None
f = flat_field(raw, self.dark, self.gain_map)
if self.maps is not None:
f = cv2.remap(f, self.maps[0], self.maps[1], cv2.INTER_LINEAR)
q = frame_quality(f)
ok_q, reasons = gate(q)
return (f if ok_q else None), reasons, q
Capture, flat field, undistort, quality gate, optional enhancement, inference. Every stage's parameters are versioned artefacts tied to a camera serial, and the gate emits a reason string on every rejection.
What this buys you
On a food-sorting deployment we reviewed, a model that scored 93% on the client's benchmark was running around 86% in the plant. No retraining was involved in closing most of the gap: the causes were conveyor vibration blurring roughly one frame in nine, and an over-eager auto-exposure reacting to sunlight through a roof light. Manual exposure, a strobe, a flat field and a focus gate recovered most of it in under a week. The last few points came from retraining — but only because the training data was finally consistent with what the camera actually produced.
The lesson we keep re-learning: specify and monitor your inputs, or you will spend your budget compensating for them in the model.
SentientSight's OpenCV consultants design and audit acquisition pipelines — lighting and lens selection, camera settings and trigger design, flat-field and colour calibration, and input-quality gating with drift telemetry — for inspection, robotics and analytics systems in production. If your accuracy numbers are worse in the field than in the lab, get in touch and we will start with the pixels.