# Exp03 Hand-Eye Calibration

> Generated from repository source files. Do not edit this page directly; edit the source file listed below and rerun `vp run docs:generate`.

> Source: `experiments/exp03-closed-loop-grasping/hand-eye-calibration.md`

## Hand-Eye Calibration Protocol

Status: `calibration binding template`

This protocol is a precondition for physical-robot trials in Experiment 3. It defines what must be recorded and validated before camera-frame detections or grasp poses are used for robot execution.

## Evidence Boundary

* `GRASP-006` supports the classical eye-on-hand calibration rationale: estimate the relative 3D pose between the camera and the robot gripper so that vision measurements can be related to robot motion.
* `GRASP-005` shows that closed-loop depth grasping still depends on camera parameters and known camera-robot calibration when converting image-space grasps to world coordinates.
* `GRASP-007` supports robot-side continuous visual servoing but does not remove the need to record project hardware, transfer limits, and dry-run validation.
* This file does not claim the SAH-BRI-Grasp hardware has been calibrated. The actual camera, robot, target, solver, station count, and acceptance thresholds remain `needs confirmation`.

## Calibration Record Identity

Every physical Exp3 session must reference a calibration record. A failed or exploratory calibration run should receive a record ID and remain in the log; do not overwrite failed attempts.

| Field | Value | Status |
| --- | --- | --- |
| calibration\_record\_id | TBD | needs confirmation |
| hardware\_inventory\_id | TBD | needs confirmation |
| threshold\_freeze\_record\_id | TBD | needs confirmation |
| protocol version | TBD | needs confirmation |
| date/time | TBD | needs confirmation |
| operator | TBD | needs confirmation |
| run mode | dry\_run / physical | needs confirmation |
| calibration status | planned / passed / failed / superseded | needs confirmation |

## Transform Convention

Define and freeze transform names before implementation:

| Transform | Meaning | Status |
| --- | --- | --- |
| `T_base_ee` | robot base to end-effector or gripper frame | needs hardware binding |
| `T_ee_cam` | end-effector or gripper frame to camera frame | calibration target |
| `T_cam_target` | camera frame to calibration target frame per station | estimated from target observation |
| `T_base_target` | robot base to calibration target frame | validation or derived quantity |

The implementation must use one convention consistently across logs, solver input, grasp planning, and visualization.

All transforms must record translation units, rotation representation, frame names, and multiplication order. If the implementation uses inverse transforms, log both the stored transform and the inverse-generation rule.

## Required Metadata

Record these before calibration:

| Field | Example / Notes | Status |
| --- | --- | --- |
| robot model | arm, controller, gripper, tool frame definition | needs confirmation |
| camera model | RGB or RGB-D, resolution, intrinsics source | needs confirmation |
| camera mounting | eye-on-hand, fixed offset, mount rigidity | needs confirmation |
| calibration target | checkerboard, circle grid, AprilTag board, or equivalent | needs confirmation |
| solver | Tsai-Lenz/OpenCV hand-eye or alternate solver | needs confirmation |
| solve station count | number of robot poses used to solve transform | needs confirmation |
| holdout station count | number of robot poses reserved for validation | needs confirmation |
| safety mode | dry-run or reduced-speed robot mode | needs confirmation |
| threshold freeze record | calibration acceptance thresholds | needs confirmation |

## Procedure

1. Verify camera intrinsics or run intrinsic calibration if intrinsics are unknown.
2. Mount the camera rigidly to the end-effector or gripper and record the tool frame definition.
3. Place the calibration target in a stable pose inside the robot workspace.
4. Generate robot stations with visible target observations and diverse translations/rotations around at least two non-colinear axes.
5. For each station, record robot pose, image/depth frame, timestamp, target detection, and target pose estimate.
6. Split stations into solve and holdout sets before fitting the transform.
7. Solve `T_ee_cam` with the selected hand-eye solver.
8. Validate on holdout stations by comparing predicted and observed target/camera poses.
9. Run a non-contact dry run: transform a known target point into the robot frame and move only to a safe pre-approach pose.
10. Lock the calibration file with date, hardware identifiers, solver name, station count, residuals, and operator notes.

## Station Log Schema

Each calibration station should have one row or JSON record with:

| Field | Required | Notes |
| --- | --- | --- |
| `calibration_record_id` | yes | links to calibration run |
| `station_id` | yes | unique within calibration record |
| `used_for` | yes | solve / holdout / rejected |
| `monotonic_ns` | yes | aligns with runtime timebase if available |
| `wall_time_iso8601` | yes | human-readable audit trail |
| `T_base_ee_ref` | yes | robot pose record or transform file |
| `image_ref` | yes | calibration image |
| `depth_ref` | conditional | required if depth is part of validation |
| `target_detection_ref` | yes | target corners/tags/pose observation |
| `T_cam_target_ref` | yes | target pose estimate |
| `observation_quality` | yes | reprojection error, blur, visibility, or detection flags |
| `rejection_reason` | yes | `none` if used |

## Calibration Output Package

Store calibration outputs under:

```text
data/calibration/<calibration_record_id>/
```

Required outputs:

| Output | Required | Notes |
| --- | --- | --- |
| `calibration_record.md` or `.json` | yes | identity, hardware IDs, status, operator notes |
| `station_log.jsonl` | yes | station schema above |
| `target_observations/` | yes | images, detected target records, optional depth |
| `transform_T_ee_cam.json` | yes | final transform, units, frame names, covariance/residuals if available |
| `validation_summary.md` or `.json` | yes | solve/holdout metrics and pass/fail |
| `dry_run_summary.md` or `.json` | yes | non-contact target checks |
| `supersedes.txt` | conditional | previous calibration IDs invalidated by this run |

## Acceptance Metrics

Threshold values remain `needs confirmation`, but the report must include:

| Metric | Purpose | Status |
| --- | --- | --- |
| target detection success rate | ensure station observations are usable | needs threshold |
| reprojection error | camera/target observation quality | needs threshold |
| solve residual | hand-eye fit quality on solve stations | needs threshold |
| heldout pose prediction error | hand-eye transform generalization | needs threshold |
| transform repeatability | compare independent calibration runs | needs threshold |
| dry-run spatial error | check robot-frame target consistency without grasping | needs threshold |
| calibration age and hardware changes | decide when recalibration is required | needs rule |

## Failure Rules

* Do not run physical grasp execution if calibration metadata is missing.
* Do not run physical grasp execution if target observations are unstable or heldout validation fails.
* Do not run physical grasp execution if `hardware_inventory_id`, `threshold_freeze_record_id`, or transform convention differs from the planned session.
* Do not use solve-station metrics alone as acceptance evidence; holdout and dry-run checks are required.
* Recalibrate after camera remounting, gripper/tool frame changes, robot controller frame changes, or unexplained grasp-frame offsets.
* Log failed calibration attempts instead of overwriting them.

## Outputs

The calibration run should produce:

* calibration transform file;
* station log;
* target observation log;
* validation summary;
* dry-run verification summary;
* operator notes and hardware identifiers.