Learning To Reach and Reaching To Learn - A Unified Approach to Path Planning and Reactive Control through Reinforcement Learning

You are cordially invited to attend the PhD Dissertation Defense of Kail FRANK on Tuesday, October 21st 2014 at 15h30 in room 351 (main building)

Abstract:
The next generation of intelligent robots will need to be able to plan reaches. Not just ballistic point to point reaches, but reaches around things such as the edge of a table, a nearby human, or any other known object in the robot’s workspace. Planning reaches may seem easy to us humans, because we do it so intuitively, but it has proven to be a challenging problem, which continues to limit the versatility of what robots can do today.
In this document, I propose a novel intrinsically motivated RL system that draws on both Path/Motion Planning and Reactive Control. Through Reinforcement Learning, it tightly integrates these two previously disparate approaches to robotics. The RL system is evaluated on a task, which is as yet unsolved by roboticists in practice. That is to put the palm of the iCub humanoid robot on arbitrary target objects in its workspace, starting from arbitrary initial configurations. Such motions can be generated by planning, or searching the configuration space, but this typically results in some kind of trajectory, which must then be tracked by a separate controller, and such an approach offers a brittle runtime solution because it is inflexible. Purely reactive systems are robust to many problems that render a planned trajectory infeasible, but lacking the capacity to search, they tend to get stuck behind constraints, and therefore do not replace motion planners.
The planner/controller proposed here is novel in that it deliberately plans reaches without the need to track trajectories. Instead, reaches are composed of sequences of reactive motion primitives, implemented by my Modular Behavioral Environment (MoBeE), which provides (fictitious) force control with reactive collision avoidance by way of a realtime kinematic/geometric model of the robot and its workspace. Thus, to the best of my knowledge, mine is the first reach planning approach to simultaneously offer the best of both the Path/Motion Planning and Reactive Control approaches.
By controlling the real, physical robot directly, and feeling the influence of the constraints imposed by MoBeE, the proposed system learns a stochastic model of the iCub’s configuration space. Then, the model is exploited as a multiple query path planner to find sensible pre-reach poses, from which to initiate reaching actions. Experiments show that the system can autonomously find practical reaches to target objects in workspace and offers excellent robustness to changes in the workspace configuration as well as noise in the robot’s sensory-motor apparatus.

Dissertation Committee:

• Prof. Jürgen Schmidhuber, Università della Svizzera italiana/IDSIA, Switzerland (Research Advisor)
• Dr. Alexander Förster, IDSIA, Switzerland (Research Co-Advisor)
• Prof. Kai Hormann, Università della Svizzera italiana, Switzerland (Internal Member)
• Prof. Rolf Krause, Università della Svizzera italiana, Switzerland (Internal Member)
• Prof. Ben Kuipers, University of Michigan, USA (External Member)
• Prof. Giorgio Metta, University of Genova, Italy (External Member)