Pi Gets a Grip on Things

In this video, Pi Robot uses his head mounted camera and range sensors to locate the target (green or red balloon) in 3-dimensional space. The coordinates of the balloon in the head-centered frame of reference are then transformed into coordinates relative to reference frames attached to each shoulder joint. From there we use spherical coordinates to figure out where to point the hands in anticipation of grasping the balloon.

The behavior demonstrated in the video takes place using five independent threads: head tracking, arm tracking (x2), grasping, and orienting.