Perception for Rock Sampling

In autonomous manipulation research at Carnegie Mellon, a robot first perceives and then grasps objects with a gripper or hand-like tool. The technology has applications in both planetary exploration and excavation on earth. To perform in these natural environments, the perception system must recognize the irregular geometry of rocks and also single out objects for manipulation whether those objects exist in cluttered or barren terrains. To this end, the perception and manipulation system picked up objects successfully in mockup test beds of sand and rock. By functioning autonomously, the perception and manipulation systems avoid the drawbacks of teleoperation-particularly for planetary exploration-where long-distance operation slows communication between the robot and its operators. The robot must view and lift objects with a minimal amount of remote human instruction.

For collecting rocks in planetary exploration, the robot has three main objectives: to sense the terrain and the objects in it, to choose the appropriate three-dimensional model with which to draw a computerized image of the terrain, and to grip the object with either a gripper or hand tool. The robot senses the terrain with a short-range sensor. By projecting light on the scene in a special sequence of patterns, the sensor computes (using triangulation) the positions of all points in the scene.

The perception system then uses three successive perception modules to build an image of the objects in the terrain. The first module, using sensor geometry, conducts a feature detection and shadow analysis of the terrain; this module produces an image of the terrain's shadows and object edges. To fill in the area of the objects, the perception system next chooses between either of two modules: the superquadric surfaces analysis, which represents the object with a three-dimensional mathematical equation, and the deformable surfaces analysis, which defines all points on the object's area. The superquadric surfaces analysis is a better representation of an object that is fairly isolated in the terrain; the deformable surfaces analysis provides a more accurate analysis of objects which are clustered together. Finally, the perception system merges images of the terrain taken from different viewpoints into one composite view. Through merging these images, the robot knows where objects are in relation to itself and to the rest of the terrain.

After choosing between the superquadric and deformable surfaces modules and merging all viewpoints, the perception system must decide which grasping tool to use: the basic gripper, which works best for lifting an isolated rock and for pulling out a rock that is partially buried; or the hand (with fingers) that can negotiate an individual or smaller rock out of a more cluttered area. To determine if the object can be lifted at all, a grasping algorithm matches the dimensions of the tool with the measurements of the rock.

Currently, the perception system relies an remote operators to decide which three-dimensional modules-superquadric or deformable surfaces-or which tool-hand or gripper-to use to grasp the object. Carnegie Mellon researchers are further developing the perception and manipulation system in robots to choose autonomously which perception and grasping methods to use. Researchers will also automate the robot's ability to single out and grasp certain types of rocks.

Point of Contact:
Martial Hebert,
Katsuchi Ikeuchi
Carnegie Mellon University
Field & Mobile Robotics Building
5000 Forbes Avenue
Pittsburgh, PA 15213-3890

Telerobotics Program page

Please email the site webmaster with any comments, criticisms or corrections for this page.
Maintained by: Dave Lavery
Last updated: May 10, 1996