The term “field robotics” was created to distinguish an emphasis on robotics in unconstrained, uncontrived settings, typically outdoors and in the full range of operational and environmental conditions: robotics in the ‘natural' world.
Field robotics first emerged in 1983 at Carnegie Mellon with the development of systems to respond to the Three Mile Island nuclear accident (Whittaker) and in autonomous off-road vehicles (Thorpe). At that time the Field Robotics Center was created. In 2008 the institute organized a major symposium marking 25 years of field robotics research (http://www.fr25.org ).
Field robotics posits that robotics problems are best solved when they are applied in the relevant field setting, validated in the environment with no simplifications or contrivances, using a field-ready (pre-commercial) fully integrated system. There are two related assumptions: that the field is the only valid setting for experimental verification of robotic algorithms, mechanisms and systems; and that through field experimentation one can gain further and deeper insight into the true nature and relative significance of fundamental problems in robotics.
The Robotics Institute is the world leader in field robotics in size of effort and funding, in breadth of research, and in all metrics of output including graduates, publications, and systems. Our work is applied to agriculture, transportation, mining, construction, energy-supply, manufacture, maintenance, space, exploration, security and defense. A distinguishing aspect of robotics at Carnegie Mellon is the number of unique robots put into extended use in the field. Typically these are full systems that employ sensing, computing, mobility, power, and control in a concerted fashion. They operate, often over months at a time, in the real world, and many times have seen their core technologies licensed, or been a part of a start-up and/or company-acquisition.
The 2007 DARPA Urban Challenge was a milestone event in field robotics. The challenge publically demonstrated that autonomous vehicles can operate reliably on varied roads, through intersections, with dynamic traffic. We believe that people's perception of what is possible, and even expected, in the future of robotics; a future that was forever transformed by this event and the attention it attracted. Responding to competitions, like the DARPA Urban Challenge, and before that the DARPA Grand Challenge, consume resources, time and people, but they catalyze advances in research and technology that go well beyond their expense.
The majority of work in field robotics occurs within the National Robotics Engineering Center, with focus and remarkable growth in applied research and commercial development, and the Field Robotics Center, with emphasis on academic research. Notable related technologies have found their way into industry via licensing to commercial partners and industry consortia or through spin-off companies (Automatika, Applied Perception, Carnegie Robotics, Proto-Innovations, RedZone and RE2).
Field robotics is a strong and growing component of the Robotics Institute. Autonomous investigation (broadly defined) of places where humans cannot or will not go will continue to be a major thrust of research in field robotics. Whether these robots are searching for life on a distant planet, securing a military installation, tracking firefighters in a burning building or investigating leaks in a sewer, we see opportunities for using reliable and intelligent agents in the world. Increasingly in the future we see field robots going on to perform specific tasks and real work. We expect that this next wave of robots will be brought into being, not just to observe, but by the need to assemble structures, haul ore, transport cargo and harvest crops. Such applications benefit from automation but require technological maturity and cost reduction before automation can become feasible.
Field robotics addresses the full range of academic and applied robotics research. Specifically since 2005 we saw significant concentration and advances in the areas of robotic/driverless vehicles, security/defense, agriculture, exploration, subterranean systems, and aerial robotics. These key areas and activities for the 2005-2010 time-span, are listed below:
In this area the RI clearly received worldwide notoriety as part of its participation and eventual win in the DARPA Urban Challenge and second and third places in the Grand Challenge. In the Urban Challenge an autonomous vehicle was required to drive through an urban setting while following the rules of the road and interacting with other vehicles. It demanded advances in sensing, planning, learning and control and, perhaps least acknowledged, in robotic systems engineering that ensured the reliability and endurance needed for success in the half-day race. The challenge was so complex that it involved the research of many our faculty (Urmson, Whittaker, Bagnell, Dolan, Hebert, Kelly, Rybski, Simmons, Singh, Stentz) and has fostered many of our graduate students, as well as attracting a new generation of faculty and students to the Robotics Institute.
Details on each the main challenges are:
- The Grand Challenge was an autonomous, 200-kilometer, driverless desert race, where CMU drove farthest and fastest in the 2004 pre-run, which led to the hotly-competed 2005 event. Carnegie Mellon's Highlander was far ahead throughout the race, but fell behind in the late miles, impaired by mechanical problems. In the end CMU's Sandstorm and Highlander claimed second and third place. Technologically, the Grand Challenge broke the barriers to high-speed, off-road navigation as it required fast planning, fast computing and new fusion algorithms, thereby establishing a new standard in robot system integration and field testing (8,000 miles of automated driving preceded the competition). Culturally, the Grand Challenge changed the world's view of what is possible with mobile robots and self-driving cars. The Grand Challenge enrolled broad participation at Carnegie Mellon, and it engaged an entirely new generation of robotics communities from around the world.
- The Urban Challenge was an autonomous, on-road driving competition for driverless cars, covering 60 miles. Robots had to obey traffic laws, negotiate other traffic and obstacles while merging, passing and navigating for the fastest finish. Carnegie Mellon's Chevy Tahoe, “Boss,” backed by General Motors and Caterpillar won the $2 million first prize. Software had to make intelligent decisions in real time based on the actions of other vehicles. Previous autonomous vehicle successes exploited structured situations such as highway and trail driving with little interaction between vehicles. This competition addressed urban driving which required cars to perform sophisticated interactions with each other, such as sharing the road, merging, parking and yielding at intersections. Technical breakthroughs included 360-degree perception, “never-give-up” planning, and behaviors to encode rules-of-the-road, resulting in very broad cross-CMU participation of multiple talented faculty (Urmson, Bagnell, Dolan, Hebert, Likhachev, Rybski, Singh, Stentz, Whittaker), and resulted in collaboration with industry partners (GM, Caterpillar) that continues through today for expansion of research and application of the same to their future product lines.
Security and Defense
In the security and defense areas, autonomous field robots achieved extremes of robustness and capability. Converging research with vehicle design, natural terrain perception, and learning control culminated in the Crusher robot (Bares, Kelly, Stentz). This robotic system achieves unprecedented mobility in rough terrain, far exceeding conventional vehicles, and demonstrates reliable autonomous driving in thousands of miles of experiment. On the smaller end of the scale, Dragon Runner (Schempf) was developed/delivered/fielded to/by the US Marine Corps in Iraq/Afghanistan, and still stands as the only single-warfighter portable and throwable rugged reconnaissance robot in the NATO arsenal.
Early agricultural projects produced first demonstrations of field crop harvesting and ornamentals (Schempf) and today are seen in commercial auto-steering systems for tractors (Stentz). Recent focus is on specialty crops, like apples, oranges and strawberries, where diverse activities for inspection, spraying, pruning and harvesting require sensing and manipulation in the wet, dirty environment of the farm (Singh, Kantor, Bergerman, Herman). Research and development has sourced from strong partnerships with industry. In addition to being technologically feasible, agricultural robots must show their economic value. Through close working relationships with growers, university extension educators, and plant scientists, industry needs are identified and addressed with a combination of science and technology, with the resulting solutions transitioned into commercial use.
Ongoing projects include Comprehensive Automation for Specialty Crops (CASC), which focuses on increasing production efficiency and decreasing labor costs in the specialty crops industry, with a special attention to apple and nursery tree production. It has led to the development of robotic utility vehicles for automation in orchards and the creation of systems that detect plant stress and disease, monitor insects, measure tree diameter, and count and size fruit prior to harvest (Singh, Bergerman, Kantor).
A more information-based project, MINDS (Managing Irrigation and Nutrients via Distributed Sensing) focuses on saving water, increasing efficiency and reducing the environmental impacts of agricultural production practices, by using sensor network data with plant physiology models for automated irrigation and nutrient management in ornamental crops. As partners in the University of Maryland-led project, RI scientists (Kantor) are using wireless sensor networks and environmental modeling to more accurately predict and apply irrigation water in nursery and greenhouse operations, and to monitor green roofs for storm-water mitigation.
Exploration, particularly for planetary surfaces, has been an area of research for over two decades (Apostolopoulos, Stentz, Simmons, Wettergreen, Whittaker) and continues as we are NASA's most significant partner in space robotics research. Applications on Earth also abound as we develop machines but more often the onboard sensing and control to enable them to localize, map, and guide action. In exploration, planning is essential to deal with incrementally discovered information about the world through noisy sensors. Several faculty members are developing methods of planning and control for robots operating under these conditions (Stentz, Kelly, Simmons, Singh, Wettergreen).
Building on RI's strong legacy in exploration, its newest endeavor pursues robotic Moon missions to explore, teach, touch and inspire a new pinnacle of human spirit and technical accomplishment. The first goal is to win Google's $20M Lunar X Prize by driving 5 kilometers, surviving lunar night(s), and visiting historical lunar hardware left behind by previous missions, particularly the Apollo 11 landing site. Future destinations include poles, craters, skylights and lava tubes to explore, prospect, and discover. Another goal is to robotically circumnavigate the Moon. RI is creating technologies, robots, spacecraft, relationships, and leaders to deliver this future. Key technologies will include sensing, control and remote operations for landing and roving. Low gravity and persistent, high-band communication are distinctive lunar advantages. Disadvantages include long, cold nights. Work to date has prototyped the rover, developed many component technologies, and designed the lander and its mission.
Another project in planetary exploration is Scarab, created to evaluate and demonstrate a combined drilling and science rover platform for lunar exploration. It has been designed to carry a 1-meter coring drill and a payload of science instruments that can analyze the abundance of hydrogen, oxygen and other materials using a unique transforming chassis (Apostolopoulos) that allows it to lay down on the ground to stabilize the drill and also to posture itself for slope ascent and descent. Lunar surface navigation uses laser scanning technologies to perceive terrain in total darkness and has demonstrated autonomous descent into craters and high-precision approach to science sites from kilometers distant (Wettergreen). Scarab has operated in Moses Lake Dunes in Washington (2007) and on Mauna Kea in Hawaii (2008).
An earth-bound, yet underwater analog to the space exploration area, is the DepthX system. The Deep Phreatic Thermal Explorer (DEPTHX) project involved FRC faculty (Kantor, Wettergreen) to create the autonomy needed to enable an underwater robot to map three-dimensional spaces like flooded caverns and mines of the Zacatón Cenote in central Mexico, seeking to understand the unique organisms that survive in this, the deepest flooded sink hole in the world. The goals of the DEPTHX project were: exploration of an unmapped underwater caves and tunnels; three-dimensional mapping of volumetric extent; modeling of environmental parameters and their gradients; characterization of localized site to identify region candidates for biological investigation; automated image and data collection with in-situ analysis for adaptive sampling.
Activities in subterranean applications of field robotics were numerous in recent years. Responding to growing attention to mine accidents, such as Quecreek near Pittsburgh, researchers have created nearly a dozen novel robots that address operation in subsurface environments, and apply robotic technologies to explore, map, rescue, and work in tunnels, mines, caves, and sewers. To enable such deployments, the design and configuration of robotic systems in subterranean mines (Whittaker, Apostolopoulos, Thayer) such as the Cave Crawler, underwater caves (Wettergreen) using DepthX, natural gas pipelines (Schempf, Browning) via the Explorer and hydrocarbon storage-tanks (Schempf) using Neptune, has been a key thrust of our faculty. At the National Robotics Engineering Center (see separate section in the report), automating mine operations on the surface for excavation and haulage (Stentz, Singh, Bares), and configuration of robots for subsurface (gold-) mines (Bares) also represent significant ongoing projects in this arena.
There is a fundamental shift underway in mining, warfare, and civil infrastructure to automate operations and remotely gather data in such areas. While the minerals mined underground provide the raw materials and power that fuel our societies and economies, these underground spaces are inherently dangerous to humans and extremely challenging for equipment and automation. Moreover, voids from underground operations and geologic formations present unique challenges to this work. Mine rescues, illicit border crossings, sewer infrastructure and cave warfare all motivate underground robot capability. Technologies required for this application area include perception, planning and hardware that are specialized for subterranean operations. Perception can exploit the advantages of natural materials and muted lighting underground, by using thermal imaging for distinguishing humans and radar for dusty/smoky areas. Modeling is quasi-tubular versus quasi-planar, since tunnels/caves are bounded by ceilings and walls, not just floor planes. Planning addresses networked corridors and restricted passages that preclude wide turns. Roof-falls, parked equipment and spontaneous events defy pre-planning, making this a very challenging research application arena. Ongoing research is improving perception, specializing robotic hardware for utility, duration and safety, and simplifying interfaces for infusion of these robots into field applications.
RI scientists develop systems that enable unmanned aerial vehicles to autonomously take-off, fly, and land in support of applications such as cargo transportation, mapping, combat medical evacuation, and reconnaissance and surveillance, among others. The required autonomy systems combine the sensing, computing, and intelligence required for safe and robust unmanned aerial flight of both fixed- and rotary-wing vehicles ranging in size from small indoor to large full-size aircraft. At the core of the work lies state-of-the-art perception sensors and algorithms that create high-fidelity, three-dimensional world maps on demand, analyze them in real time, and generate semantic representations of the environment that the aircraft control system uses to generate flight plans; and planning methods and algorithms that reason upon the semantic representations to guide the aircraft along safe, obstacle- and collision-free trajectories. The systems fly on-board actual unmanned aerial vehicles in urban, forested, and riverine environments.
Unmanned aerial vehicles are mostly fixed-wing, invaluable for reconnaissance, and typically fly blind on preplanned routes at a high altitude, taking off and landing at prepared sites. In contrast, an unmanned rotorcraft is required to fly at low altitudes, taking off and landing at unprepared sites. An important part of aerial robotics work at RI aims to create self-sufficient rotorcraft which can fly and do their job without external aids or extensive communication networks, and are arguably the most advanced aerial research platforms in the world. From the start, Carnegie Mellon focused on the weaknesses of relying on external means for flight. A key accomplishment over the past five years has been the development of self-aware rotorcraft able to avoid obstacles and locate acceptable landing sites from onboard sensors (Amidi, Kanade, Singh) solely based on visual feedback from their environment. These efforts are recognized by the UAV community as the state-of-the-art image-based methodology for autonomous flight in GPS-denied environments. Additional efforts in multi-UAV flight-team planning/coordination/control (Scerri) for effective mission-execution have also been an important focus for increased efficient use of UAV assets.
One of the systems developed in this area is AeroScout. It allows for safe operation of unmanned rotorcraft at low-altitude while avoiding small obstacles such as wires quickly, using a reactive algorithm that avoids obstacles after training by observing human behavior; the algorithm was demonstrated with speeds up to 10 m/s in more than 1,000 obstacle avoidance runs.
In collaboration with the world's largest aircraft manufacturer (Boeing), our scientists have enabled a full-size helicopter to autonomously land at unimproved landing sites for casualty evacuation. They (Singh et al.) developed a custom 3D laser scanner and algorithms that were able to evaluate landing sites in real-time and land the Boeing Unmanned Little Bird helicopter without human assistance.
The RI significantly advanced the frontiers of field robotics over the past six years. Technological and research challenges that will need to be addresses over the coming years, include (i) sensor accuracy and robustness, (ii) actuator power/weight ratios, (iii) battery energy density, (iv) communication-bandwidth and -range, and (v) human interface effectiveness. Barriers to adoption of field robotic technology include system safety, durability and cost as well as cultural perceptions of robots and their role. Yet we see sustainable agriculture that replaces monoculture with persistent and intensive cultivation, commercial space enterprise with dozens of small companies delivering payloads, creating intelligent satellites and robotic systems and driverless racing at high speed that fearlessly outperforms human driving. We believe that recent accomplishments, like the Urban Challenge, are making valuable progress and world-class impact in the adoption of field robotics.
Table of Contents
- Robotics Institute Research Guide
- A robust approach to high-speed navigation for unrehearsed desert terrain
- Autonomous Driving in Urban Environments: Boss and the Urban Challenge
- Comprehensive Automation for Specialty Crops: Year 1 results and lessons learned
- Junior: A Robot for Outdoor Container Nurseries
- Segmented SLAM in Three-Dimensional Environments
- Autonomous Exploration and Mapping of Flooded Sinkholes
- Design and field experimentation of a prototype Lunar prospector
- Towards Topological Exploration of Abandoned Mines
- Explorer: Untethered Real-Time Gas Main Assessment Robot System
- Flying Fast and Low Among Obstacles: Methodology and Experiments
- Online Assessment of Landing Sites
- Comprehensive Automation for Specialty Crops: Year 1 results and lessons learned
- Deployment of Wireless Sensor Networks for Irrigation and Nutrient Management in Nursery and Greenhouse Operations
- Houdini: Site and Locomotion Analysis-driven Design of an In-tank Mobile Cleanup Robot
- Ultra-rugged Soldier-Robot for Urban Conflict Missions