HomeLearnCoursesHackathonsAccount
Imitation Learning & Learning from Demonstration
What Imitation Learning Is, and Why Skip the Reward Function · 1/2

Learning from examples instead of learning from trial and error

Reinforcement learning asks a robot to discover good behavior on its own, guided only by a scalar reward signal that tells it how well it did after the fact. That works, but it requires someone to design a reward function first, and reward design is genuinely one of the hardest parts of applying RL to a real task. A reward that's too sparse gives the robot almost nothing to learn from. A reward that's too dense or poorly shaped invites reward hacking, where the robot finds a technically-high-scoring behavior that isn't the behavior you actually wanted. Getting a reward function right often takes as much engineering effort as the learning problem itself.

Imitation learning sidesteps that problem by changing what the robot learns from. Instead of a reward signal, you provide examples: recordings of a human (or another competent controller) actually performing the task. The learning problem becomes 'reproduce this behavior' rather than 'discover a behavior that scores well against this function I designed.' For tasks where you can demonstrate the behavior but struggle to formalize what makes it good, like folding a towel, pouring a liquid, or navigating a cluttered kitchen, imitation learning gives you a way to transmit that competence directly, without ever writing down a reward.