ABSTRACT
Before developing mobile manipulation, this chapter first considers combining perception and manipulation. Combining perception and manipulation also requires hand-eye calibration, and thus extrinsic camera calibration is discussed. Perception and manipulation are combined to enable vision-based manipulation. The object-finder action server and object-grabber action server are used together, coordinated by a command bundler, for this exercise. For vision-based manipulation to work, it is important that the imaging source is calibrated to the robot. A Kinect sensor is retrofit to the Baxter robot (model and/or physical robot). A model file that achieves this is baxteronpedestal.xacro in the package baxtervariations. The chapter shows that rviz can be used to visualize the quality of extrinsic camera calibration, and it can be used to perform calibration interactively. It also shows how Even greater flexibility can be realized if the manipulator is mobile. The example command bundler action server and the corresponding action client nodes presented illustrate how to integrate perception and manipulation.
