Active Articulation Model Estimation
When operating in human environments, robots must frequently cope with multiple types of uncertainty such as systematic and random perceptual error, ambiguity due to a lack of data, and changing environmental conditions. While methods like Bayesian reasoning allow robots to make intelligent decisions under uncertainty, in many situations it is still advantageous, or even necessary, to actively gather additional information about the environment. The paradigm of interactive perception aims to use a robot’s manipulation capabilities to gain the most useful perceptual information to model the world and inform intelligent decision making.
In this work we introduce a particle filter-based approach to representing and actively reducing uncertainty over articulated motion models. The presented method provides a probabilistic model that integrates visual observations with feedback from manipulation actions to best characterize a distribution of possible articulation models. We evaluate several action selection methods to efficiently reduce the uncertainty about the articulation model. The full system is experimentally evaluated using a PR2 mobile manipulator. Our experiments demonstrate that the proposed system allows for intelligent reasoning about sparse, noisy data in a number of common manipulation scenarios.
More detailed description of this work can be found in our paper.
Autonomous Regrasping using Reinforcement Learning
Interactive Grasping using Biomimetic Tactile Sensors
One of the ambitious goals in my research is to develop a method that uses interactive perception for grasping. We accomplished a first step towards this goal where we are able to detect and classify slip based on the tactile sensors (please see below). Stay tuned! There are more things to come!
We introduce and evaluate contact-based techniques to estimate tactile properties and detect manipulation events using a biomimetic tactile sensor. In particular, we estimate finger forces, and detect and classify slip events. In addition, we present a grip force controller that uses the estimation results to gently pick up objects of various weights and texture.
The estimation techniques and the grip controller are experimentally evaluated on a robotic system consisting of Barrett arms and hands. Our results indicate that we are able to accurately estimate forces acting in all directions, detect the incipient slip, and classify slip with over 80% success rate. For more details on our approach please take a look at our Humanoids paper.
Interactive Segmentation for Textured Objects
This project focuses on a novel object segmentation approach for autonomous service robots acting in human living environments. The proposed system allows a robot to effectively segment textured objects in cluttered scenes by leveraging its manipulation capabilities.
In this interactive perception approach, 2D-features are tracked while the robot actively induces motions into a scene using its arm. Thereby, the robot autonomously infers appropriate arm movements which can effectively separate objects. The resulting tracked feature trajectories are assigned to their corresponding object by using a novel randomized clustering algorithm, which samples rigid motion hypotheses for the a priori unknown number of scene objects.
We evaluated the approach on challenging scenes which included occluded and reflective objects, as well as objects of varying shapes and sizes. More detailed description can be found in our paper.
Interactive Segmentation for Textureless Objects
This project focuses on a textureless object segmentation approach for autonomous service robots acting in human living environments. The proposed system allows a robot to effectively segment textureless objects in cluttered scenes by leveraging its manipulation capabilities.
In our pipeline, the cluttered scenes are first statically segmented using state-of-the-art classification algorithm and then the interactive segmentation is deployed in order to resolve this possibly ambiguous static segmentation. In the second step the RGBD (RGB + Depth) sparse features, estimated on the RGBD point cloud from the Kinect sensor, are extracted and tracked while motion is induced into a scene. Using the resulting feature poses, the features are then assigned to their corresponding objects by means of a graph- based clustering algorithm. In the final step, we reconstruct the dense models of the objects from the previously clustered sparse RGBD features.
We evaluated the approach on a set of scenes which consist of various textureless flat (e.g. box-like) and round (e.g. cylinder-like) objects and the combinations thereof. More detailed description is available in our paper.
Interactive Object Recognition
Object recognition is a key component of service robots for finding and handling objects. Current state-of-the-art object recognition systems recognize objects based on static images. However, these systems prove limited in cases when objects are in ambiguous orientations or distinctive features are hidden, e.g., due to the pose of the object.
In this work we introduce a probabilistic method for choosing object manipulation actions to optimally reveal information about objects in a scene based on robot’s observations. To the best of our knowledge, the problem of interactive object recognition has not been addressed before. Our approach determines the optimal action for a robot to interact with objects and adjust their pose to reveal discriminative features for determining their identity. In the ambiguous book example (see Fig. on the left), this means flipping the book over and observing the cover, which results in more confident recognition. Our method is based on a probabilistic graphical model for feature-based object and pose recognition. By inferring posterior distributions of object probabilities conditioned on all previous actions and observations, our approach enables a robot to select the optimal action to reduce the uncertainty of the object.
This is our initial work on this problem (IROS 2014 workshop paper) and more details will be provided soon.
Multi-Sensor Fusion with Measurement Validation and Seamless Sensor Suite Switching
Fusing data from multiple sensors on-board a mobile platform can significantly augment its state estimation abilities and enable autonomous traversals of different domains by adapting to changing signal availabilities. However, due to the need for accurate calibration and initialization of the sensor ensemble as well as coping with erroneous measurements that are acquired at different rates with various delays, multi-sensor fusion still remains a challenge.
In this work, we introduce a novel multi-sensor fusion approach for agile aerial vehicles that allows for measurement validation and seamless switching between sensors based on statistical signal quality analysis. Moreover, it is capable of self-initialization of its extrinsic sensor states. These initialized states are maintained in the framework such that the system can continuously self-calibrate. We implement this framework on-board a small aerial vehicle and demonstrate the effectiveness of the above capabilities on real data. As an example, we fuse GPS data, ultra-wideband (UWB) range measurements, visual pose estimates, and IMU data. Our experiments demonstrate that our system is able to seamlessly filter and switch between different sensors modalities during run time.
Observability-Aware Trajectory Optimization
We study the nonlinear observability of a system’s states in view of how well they are observable and what control inputs would improve the convergence of their estimates. We use these insights to develop an observability-aware trajectory-optimization framework for nonlinear systems that produces trajectories well suited for self-calibration.
Common trajectory-planning algorithms to generate motions that lead to an unobservable subspace of the system state, causing suboptimal state estimation. We address this problem with a method that reasons about the quality of observability while respecting system dynamics and motion constraints to yield the optimal trajectory for rapid convergence of the self-calibration states (or other user-chosen states).
Experiments performed on a simulated quadrotor system with a GPS-IMU sensor suite demonstrate the benefits ofthe optimized observability-aware trajectories when compared to a covariance-based approach and multiple heuristic approaches. Our method is ~80x faster than the covariance-based approach and achieves better results than any other approach in the self-calibration task. We applied our method to a waypoint navigation task and achieved ~2x improvement in the integrated RMSE of the global position estimates and ~4x improvement in the integrated RMSE of the GPS-IMU transformation estimates compared to a minimal-energy trajectory planner.
Cooperative Control of Multiple Quadrotors
We consider the cooperative control of a team of robots to estimate the position of a moving target using onboard sensing. In particular, we do not assume that the robot positions are known, but estimate their positions using relative onboard sensing.
Our probabilistic localization and control method takes into account the motion and sensing capabilities of the individual robots to minimize the expected future uncertainty of the target position. It reasons about multiple possible sensing topologies and incorporates an efficient topology switching technique to generate locally optimal controls in polynomial time complexity.
Simulations show the performance of our approach and prove its flexibility to find suitable sensing topologies depending on the limited sensing capabilities of the robots and the movements of the target. Furthermore, we demonstrate the applicability of our method in various experiments with single and multiple quadrotor robots tracking a ground vehicle in an indoor environment. More detailed description is available in our paper.
Occlusion-Aware 3D Tracking
We introduce an optimization-based control approach that enables a team of robots to cooperatively track a target using onboard sensing. In this setting, the robots are required to estimate their own positions as well as concurrently track the target. Our probabilistic method generates controls that minimize the expected uncertainty of the target. Additionally, our method efficiently reasons about occlusions between robots and takes them into account for the control generation.
We evaluate our approach in a number of experiments in which we simulate a team of quadrotor robots flying in three-dimensional space to track a moving target on the ground. We compare our method to other state-of-the-art approaches represented by the random sampling technique, lattice planning method, and our previous method. Our experimental results indicate that our method achieves 4 times smaller maximum tracking error and 2.5 times smaller average tracking error than the next best approach in the presented scenario.