Hand-eye calibration and 6-DoF pick poses in OpenCV 5: from calibrateHandEye to a tolerance you can quote

Vision projects that end in a robot moving fail differently from vision projects that end in a dashboard. Detection can be perfect — the part is found in every frame, the mask is crisp — and the gripper still misses by 6 mm and crushes the part. When that happens the model is rarely the problem. The problem is that nobody ever established, numerically, the transform between what the camera sees and where the robot's tool actually is.

This tutorial walks the full chain for a pick-and-place cell built on OpenCV 5: capturing a hand-eye dataset, solving calibrateHandEye, estimating 6-DoF object pose with solvePnPRansac or a point-cloud fit, converting that pose into a robot base-frame target, and — the part most teams skip — measuring how much error is left so you can tell an integrator "we are good to ±1.5 mm at 3-sigma" instead of "it looks about right".

If you have not yet nailed intrinsics, read our camera calibration tutorial first. Everything below assumes an honest camera matrix and distortion model. Hand-eye calibration cannot rescue bad intrinsics; it inherits them.

1. The frames, written down once

Almost every hand-eye argument on a shop floor comes from two people using the same word for different frames. Fix the notation before you write code. We use T_a_b to mean "the transform that takes points expressed in frame b into frame a".

• base — the robot base frame. Everything the controller accepts as a target is expressed here.
• gripper — the tool flange (sometimes "TCP", sometimes "end effector"; they are not always the same point, ask).
• cam — the camera optical frame: +Z out of the lens, +X right, +Y down. This is OpenCV's convention and it is not ROS's optical convention by accident — ROS also defines _optical_frame this way, but the parent link is usually X-forward. Mixing them is the single most common source of a pose that is right but rotated 90°.
• target — the calibration board or the object being picked.

Two rig types, two unknowns:

• Eye-in-hand: camera bolted to the wrist. Unknown is T_gripper_cam (constant), and T_target_base happens to also be constant.
• Eye-to-hand: camera on a tripod watching the cell. Unknown is T_base_cam (constant), and the board is mounted on the gripper.

OpenCV solves both with the same function; you just feed it different inputs. Get this wrong and the residuals will be large and mysterious.

2. Capturing a dataset that can actually be solved

cv::calibrateHandEye solves the classic AX = XB problem. The mathematics needs rotational diversity: if all your poses differ only by a translation, or all rotations share an axis, the system is degenerate and the solver will happily return a confident, wrong answer.

A capture protocol that works:

1. Jog the robot to 15–20 poses (12 is a bare minimum; more than ~25 has diminishing returns).
2. Between poses, change orientation by at least 20–30° about different axes. Rolling the wrist about the same axis every time is the classic failure.
3. Keep the ChArUco board filling roughly a third to a half of the frame, and vary the standoff distance by ±30%.
4. At every pose, stop moving, wait for the controller to settle, then grab the frame and read the robot pose in the same beat. Rolling-shutter blur and a 40 ms timestamp skew both show up as rotation error later.
5. Record the robot pose from the controller in the pose format you verified — many controllers report Euler angles in degrees with a non-obvious order (ZYX vs ZYZ), or quaternions ordered w,x,y,z while your library expects x,y,z,w.

Recording, in Python:

import cv2, numpy as np

board = cv2.aruco.CharucoBoard(
    (11, 8), 0.020, 0.015,
    cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_5X5_250))
detector = cv2.aruco.CharucoDetector(board)

def target_pose(gray, K, dist):
    ch_corners, ch_ids, _, _ = detector.detectBoard(gray)
    if ch_ids is None or len(ch_ids) < 8:
        return None
    obj_pts, img_pts = board.matchImagePoints(ch_corners, ch_ids)
    ok, rvec, tvec = cv2.solvePnP(
        obj_pts, img_pts, K, dist, flags=cv2.SOLVEPNP_ITERATIVE)
    return (rvec, tvec) if ok else None

Note the modern object API — CharucoDetector.detectBoard and board.matchImagePoints. The 4.x free functions (cv2.aruco.interpolateCornersCharuco, estimatePoseCharucoBoard) are gone in OpenCV 5, and the ArUco code now lives in objdetect, not contrib.

3. Solving the hand-eye transform

For an eye-in-hand rig, OpenCV wants gripper-to-base transforms and target-to-camera transforms:

R_g2b, t_g2b = [], []   # from the robot controller, per pose
R_t2c, t_t2c = [], []   # from solvePnP on the board, per pose

R_cam2gripper, t_cam2gripper = cv2.calibrateHandEye(
    R_gripper2base=R_g2b, t_gripper2base=t_g2b,
    R_target2cam=R_t2c,   t_target2cam=t_t2c,
    method=cv2.CALIB_HAND_EYE_PARK)

For an eye-to-hand rig with the board on the wrist, invert the robot poses before you pass them in (feed base-to-gripper) and you get T_base_cam out. Writing a tiny invert(R, t) helper and being explicit beats trying to remember which argument is pre-inverted.

Method choice matters less than people expect. CALIB_HAND_EYE_PARK and ..._TSAI are fast closed-form solutions and are fine when your data is clean. ..._DANIILIDIS (dual quaternions) solves rotation and translation jointly and tends to be the most robust when translations are small. Run all five, compare residuals, and if they disagree by more than a millimetre or so, your dataset is the problem, not the algorithm.

If your robot's own tool-flange calibration is suspect, cv2.calibrateRobotWorldHandEye solves for the world-to-base and gripper-to-camera transforms together, which absorbs some of that error.

4. Validate before you trust it

calibrateHandEye returns no residual. You must compute one yourself. The standard check: for every pose i, the board's position in the base frame should be identical, because the board never moved.

def compose(R, t):
    T = np.eye(4); T[:3, :3] = R; T[:3, 3] = t.ravel(); return T

X = compose(R_cam2gripper, t_cam2gripper)
pts = []
for (Rg, tg), (Rt, tt) in zip(zip(R_g2b, t_g2b), zip(R_t2c, t_t2c)):
    T_target_base = compose(Rg, tg) @ X @ compose(Rt, tt)
    pts.append(T_target_base[:3, 3])

pts = np.array(pts)
print("spread mm:", (pts.std(axis=0) * 1000).round(3),
      "max dev mm:", (np.abs(pts - pts.mean(0)).max() * 1000).round(3))

Rules of thumb from cells we have commissioned, with a decent 5 MP camera at a 500 mm standoff:

Spread (max deviation)	Verdict
< 1 mm	Good. Proceed.
1–3 mm	Usable for forgiving grippers; recapture if you need precision.
3–10 mm	A frame convention or unit error, or too little rotational diversity.
> 10 mm	Something is inverted. Do not tune — re-derive.

Also sanity-check the translation you got out. In an eye-in-hand rig you can measure camera-to-flange offset with calipers to a few millimetres. If t_cam2gripper says 180 mm and the tape says 60 mm, stop.

Two errors account for most of the bad cases we are called in to fix: millimetres versus metres (the board square size in metres, the robot reporting millimetres — an exact factor of 1000 in translation with perfect rotation is the fingerprint), and an Euler-order mismatch in the robot pose parser, which gives good translation and nonsense rotation.

5. From detection to a 6-DoF pick pose

With X solved, object pose flows straight to the controller.

Planar or known-geometry parts. Get ≥4 correspondences between a CAD model and the image — corners, holes, fiducials, or keypoints from a trained model — and use solvePnPRansac with SOLVEPNP_SQPNP, which is the well-behaved general solver in modern OpenCV:

ok, rvec, tvec, inliers = cv2.solvePnPRansac(
    model_pts_3d, image_pts_2d, K, dist,
    flags=cv2.SOLVEPNP_SQPNP, reprojectionError=2.0, confidence=0.999)
rvec, tvec = cv2.solvePnPRefineLM(
    model_pts_3d[inliers], image_pts_2d[inliers], K, dist, rvec, tvec)

R, _ = cv2.Rodrigues(rvec)
T_obj_cam = compose(R, tvec)
T_obj_base = compose(R_g2b_now, t_g2b_now) @ X @ T_obj_cam

Watch for pose ambiguity: a small, near-planar, nearly symmetric part has two PnP solutions that both reproject well and differ by a flip. solvePnPGeneric returns all candidates with their reprojection errors — if the two best are within ~20% of each other, the pose is ambiguous and you should disambiguate with a second viewpoint or a depth reading rather than picking the first solution.

Bin picking / freeform parts. For depth-sensor pipelines, feed segmented points into a coarse global registration and refine with ICP. OpenCV 5's 3d module ships cv::ICP and a point-pair-feature matcher (formerly surface_matching in contrib); many teams pair OpenCV's 2D front end with Open3D for the registration stage. Either way the output is still T_obj_cam, and the same hand-eye composition applies.

Grasp frame ≠ object frame. The pose you send is T_obj_base @ T_grasp_obj, where T_grasp_obj is a fixed offset defined in CAD (approach along −Z, 15 mm above the grasp point, and so on). Keep it in a config file, not scattered through code, and give it a sign convention comment.

6. Error budget, so you can quote a tolerance

Nobody should sign off a cell on vibes. Add the contributions in quadrature:

• Intrinsics — reprojection RMS of ~0.2 px at 500 mm with a 4 mm lens on a 3 µm pixel is roughly 0.2 mm of lateral error.
• Hand-eye rotation — this is the sneaky one. A 0.3° rotation error becomes 500 mm × sin(0.3°) ≈ 2.6 mm at a 500 mm standoff. Rotation error scales with distance; translation error does not. Mount the camera close, or accept the penalty.
• Robot repeatability — the datasheet number (often ±0.03 mm) is repeatability, not accuracy. Absolute accuracy can be 10× worse across the workspace unless the arm has been mastered.
• Object pose estimation — measure it: place the part at ten known positions on a jig and record the residuals.
• Thermal drift — a rigidly mounted camera can move tens of microns per °C on a steel frame; over an 8-hour shift on an unheated floor that is real. Re-verify at shift start with a single fiducial check rather than a full recalibration.

Write the resulting budget into the acceptance criteria before build. "±1.5 mm, 3-sigma, over the 600 × 400 mm work area, at 20–30 °C" is a specification. "Accurate" is an argument waiting to happen.

7. Keeping it honest in production

• Daily fiducial check. One ArUco marker at a fixed jig position, one frame at shift start. If the computed base-frame position moves more than your threshold, alarm rather than drift. This takes an afternoon to build and catches nudged cameras, loosened mounts and lens knocks.
• Version the transforms. X, the intrinsics and T_grasp_obj are deployment artefacts. Store them with the camera serial number, lens, date and the residual you measured. When a cell misbehaves 14 months later, the first question is always "which calibration is on this machine?"
• Log the inputs to every pick. Frame, detected keypoints, pose, inlier count. A pick failure with no frame stored is unfixable.
• Re-run hand-eye after any mechanical event — a camera remount, a lens change, a crash, a gripper swap. It takes 20 minutes with a scripted capture and it removes an entire class of "the robot has been weird since Tuesday" tickets.

Where teams usually go wrong

Ranked by how often we see it: frame-convention mix-ups between the vision stack and the controller; a hand-eye dataset with too little rotational diversity; unit mismatches; trusting robot datasheet accuracy; and — most expensively — treating calibration as a one-off commissioning step rather than a monitored, versioned artefact of the running system.

None of this is exotic. It is the difference between a demo and a cell that runs a shift.

---

Building a vision-guided robot cell? SentientSight places senior OpenCV engineers who have commissioned pick-and-place, bin-picking and inspection cells end to end — calibration, pose estimation, controller integration and the acceptance testing that goes with them. Get in touch with your part geometry, cycle time and tolerance and we will tell you honestly whether the accuracy you need is reachable with the camera you have.