The Mars Pathfinder Sojourner rover used a combination of stereo ranging
and a laser light-striper to give ground operators a rich view of the
terrain it travelled through, and allow limited autonomy. The Mars
Surveyor 2003 and 2005 missions, to which the Mars Autonomy Project
plans to contribute, will have similar sensing and much greater
We use a stereo calibration and matching algorithm developed by the Machine Vision and
Tracking Sensors Group at JPL to convert a pair of greyscale images
from our cameras into a cloud of (x,y,z) points on the surface of the
terrain ahead. Some of the properties that distinguish stereo ranging
from other techniques are:
- It is very computationally intensive and returns
large data sets.
- Lack of texture in the terrain or unusual lighting conditions
can lead to gaps in the data.
- Precise camera calibration can allow much more efficient matching
Our traversability mapper module processes range data to generate
traversability maps which rate how easy it is to drive through
different points. These maps are passed on to the local obstacle avoidance and global path planning modules which determine the
path the rover will take.
Click for a full-size image
Stereo ranging begins with camera calibration. We use a cube with black
dots laid out in a square grid. During calibration, a simple image
processing algorithm finds the centroids of the dots with some help from
a human operator. The positions of the dots let us estimate:
- Intrinsic parameters, which relate to the camera itself:
the properties of the CCD and the lens. Some of these, like the focal
length and size of CCD pixels, are known in advance. The radial lens
distortion, however, needs to be calculated.
If the camera had no radial distortion, the image of each plane of the
cube would be a perspective transform of the grid (in particular, the
dots would be in straight lines). By measuring the deviations from the
expected grid, we can come up with a best-fit model of the
distortion and correct for it.
- Extrinsic parameters, which describe the relative position
and orientation of the individual cameras to each other and to the
world. Because the actual size of the cube is precisely known, the
disparity between dot positions in the left and right images lets us
infer the length of the stereo baseline. We can also correct for
discrepancies in the focal axis and "up" direction of the cameras.
Click for the full-size stereo pair
The Matching Process|
Stereo ranging relies on finding the same object in different positions
in the left and right camera images. From the disparity between the
positions one can infer the range to the object. The image at the left
is a map of the disparity of matched objects in the pair above. Shades
of green have the lowest disparity (longest range), fading all the way
to white, which has highest disparity (shortest range).
You can try doing your own matching on the stereo pair shown above.
Look at the pair and pretend you are trying to focus on an object far
behind your monitor screen, until the left and right images appear to
fuse in the middle. Once you have the middle image in sharp focus, you
should be able to perceive depth in the scene like our stereo algorithm.
You may have better luck with the full-size
image, moving your head at least 3-4 feet from the monitor.
After we have ranges for points on the terrain ahead at various
positions in the image, some trigonometry lets us generate (x,y,z)
points in the rover's coordinate frame. The image at the lower left is
a snapshot from a 3D browser looking at such a point cloud. The black
patches are "shadows" behind the rocks, where no data could be collected
because the terrain was obscured.
For navigation purposes, we want to tag each position in the map with a
measure of how hazardous it is. This information is useful both for
local obstacle avoidance and global path planning.|
Using local least-squares plane-fitting on range data, the
traversability mapper calculates the expected roll and pitch of the
vehicle at each map grid cell in the sensor footprint. The residual
error of the plane fit is used as an estimate of the small-scale
roughness of the terrain.
With its knowledge of the abilities of the mobile platform, like maximum
allowable roll and the chassis clearance, the mapper assigns a
traversability score to each position in the map grid. The
traversability map is the basis on which our obstacle avoidance and path
planning algorithms make their navigation decisions.|
The density of range points in each grid cell provides a measure of
certainty for the traversability score. The different modules
respond to uncertainty in different ways. The path planner
optimistically assumes that unknown areas are clear of hazards (allowing
it to plan paths through unexplored areas), while the local obstacle
avoidance system will not allow the vehicle to move into unknown terrain.
A sample traversability map is shown above. Traversability fades from
best (green) to worst (red). Certainty fades from highest (bright red
or green) to lowest (black).
Last modified: $Date: 1999/06/28 18:04:32 $