# GRASP-007: Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection > Generated from a local paper card. Do not edit this page directly; edit the source card and rerun `vp run docs:generate`. > Internal wiki boundary: paper-card evidence and user-provided PDF downloads are available for private reading. Extracted full-paper text is not published. > Source: `library/paper_cards/GRASP-007.md` ## Paper Access * Internal PDF: Download Paper * DOI / official page: [10.1177/0278364917710318](https://doi.org/10.1177/0278364917710318) * Deployment boundary: these PDF links are intended only for a private/protected internal wiki. ## GRASP-007: Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection ## Metadata * ID: GRASP-007 * Title: Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection * Year: 2018 * DOI / URL: 10.1177/0278364917710318 * Local PDF: see Paper Access section above * Text artifact: local-only path withheld from docs site * Review status: `extracted` ## Study Type * Track: Robot Vision / Grasping / Calibration * Task: Vision-guided robotic grasping with learned hand-eye coordination and large-scale self-supervised data collection. * Participants or dataset: First experiment collected about 800,000 grasp attempts over two months using 6-14 robotic manipulators; second experiment collected over 900,000 grasp attempts over about four months using Kuka IIWA robots and roughly 1100 objects. * Hardware: Multiple 7-DOF manipulators and Kuka IIWA robots with two-finger grippers and monocular camera views. * Channels or sensors: Monocular RGB images for the learned controller; some baselines use depth sensing and calibration. ## Methods * Paradigm: Self-supervised grasp attempt collection, CNN grasp-success prediction, and continuous visual servoing. * Signal processing or model: CNN predicts grasp success probability from current/pregrasp monocular images and task-space motion commands; cross-entropy method optimizes candidate commands; inverse kinematics executes vertical pinch grasps. * Training/calibration: Large-scale self-supervised labels from physical grasp attempts; the learned closed-loop method is designed to compensate for camera/robot offsets rather than requiring precise hand-eye calibration. * Online/offline: Online closed-loop grasp servoing, with the controller running around 2-5 Hz. ## Results * Metrics: Failure rate under with-replacement, without-replacement, and transfer/generalization evaluations. * Main findings: In the first evaluation, the learned method had failure rates of 10.0%, 17.5%, and 17.5% over first 10/20/30 grasps without replacement. With replacement, failure rate was 20%, compared with random 69%, hand-designed 35%, and open-loop 43%. Transfer results report a best average failure rate of 22.82 +/- 5.3% for joint training with 2.7M non-Kuka images plus 8M Kuka images. * Reported limitations: Failures include height-control heuristics and maximum servo steps, gripper wear/deformation, nearby-object ambiguity and occlusion, lack of depth, objects stuck in corners or difficult poses, limited transfer to new robots without additional data, and the need for very large datasets. ## Relevance To This Project * Supports: Robot-side continuous visual feedback, learned grasp-success prediction, and self-supervised grasp data as references for low-level autonomy. * Conflicts with: Does not include YOLO candidate selection, EEG/SSVEP/MI input, or shared-control BRI arbitration. * Design implication: `SAH-BRI-Grasp` can cite this as evidence for robot-side visual servoing and grasp-learning constraints, while keeping BCI intent selection and project hardware validation separate. ## Extracted Evidence | Claim | Status | Evidence Note | Page/Section | | --- | --- | --- | --- | | Large-scale self-supervised robot data can train a closed-loop grasping controller. | verified | The paper reports about 800,000 grasp attempts in the first experiment and over 900,000 additional attempts in the Kuka experiment. | Data-collection sections | | Learned visual servoing can improve grasp performance over simple baselines. | verified | With replacement, the learned method reported 20% failure versus random 69%, hand-designed 35%, and open-loop 43%. | Evaluation tables | | Continuous visual servoing can reduce dependence on precise hand-eye calibration but does not remove project validation needs. | inferred | The method is designed to compensate for offsets, but it still depends on specific robot/camera/data conditions and does not test SAH-BRI-Grasp hardware. | Method and project synthesis | | The study supports robot-side autonomy, not EEG-based intent decoding. | inferred | The paper contains no BCI, SSVEP, MI, or shared-autonomy BRI evaluation. | Project synthesis | ## Open Questions * How should BCI-selected object intent constrain the learned or geometric grasp controller? * Are YOLO boxes sufficient to define the allowed grasp region? * How much local robot data would be required for this project's hardware? * Should depth or stereo be added to reduce known failure cases from lack of depth and occlusion?