HomeLearnCoursesHackathonsAccount
Imitation Learning & Learning from Demonstration
Where Imitation Learning and RL Complement Each Other · 1/2

Imitation as a warm start, not a final answer

In practice, imitation learning and reinforcement learning are rarely treated as an either/or choice on real robot systems. A common and effective pattern is to use imitation learning first, to get a policy to a reasonable starting point quickly and cheaply, and then use reinforcement learning afterward to refine that policy further. Starting RL from scratch on a robot task, with a randomly initialized policy that has to stumble into any successful behavior at all before it can start improving, can be extremely slow and, on a physical robot, can involve a lot of undesirable flailing around before anything useful is discovered.

Initializing with a behaviorally-cloned policy (or one trained via DAgger or IRL) sidesteps that cold-start problem. The policy already does something roughly sensible from the very first RL rollout, because it's already imitating a demonstrator who could do the task. RL then only needs to explore around that reasonable starting point and refine it, rather than discovering competent behavior from nothing. This is both faster and safer in practice, since the robot spends less time in the exploratory, poorly-performing regime that pure RL from scratch requires.