DAgger: getting the expert to label the states the policy actually visits
DAgger (Dataset Aggregation) attacks the distribution shift problem directly, by changing what data the policy gets trained on. Instead of training once on a fixed set of expert trajectories, DAgger iterates: run the current policy on the actual task, observe the states it visits (including the slightly-off-path ones behavioral cloning never handled well), have the expert label the correct action for those specific states, and add those newly labeled states into the training set. Then retrain, and repeat. Over enough iterations, the training data comes to include not just the expert's original path but the states the learned policy tends to drift into, along with the expert's corrections for them.
This directly targets the root cause from the previous lesson: the policy failed on off-path states because it had never seen labeled corrections for them. DAgger manufactures exactly that missing data. The cost is that it requires an expert available for repeated, ongoing labeling throughout training, not just a one-time batch of demonstrations upfront, which is a heavier and more expensive process than plain behavioral cloning. It also requires the expert to be willing and able to label a live policy's states, which is easy if the expert is a human at a console but harder if the original demonstrations came from teleoperation sessions that are no longer running.