Kinesthetic teaching and teleoperation
The most direct way to collect a demonstration is kinesthetic teaching: a human physically grabs the robot's arm and guides it through the motion, while the robot logs its own joint positions and sensor readings along the way. Because the robot is recording its own actual state and actuator commands, there's no gap between what was demonstrated and what the robot's controller needs to reproduce. The downside is that it only works for robots a human can safely and comfortably manhandle, generally slower or lighter arms, and it's tedious to scale to many demonstrations or complex whole-body motions.
Teleoperation instead has a human control the robot remotely, through a joystick, a haptic controller, a VR headset, or a puppeteering rig that mirrors human arm motion onto the robot's arm. The robot performs the task itself rather than being physically dragged through it, which makes teleoperation viable for robots that would be unsafe or impractical to touch directly, and for tasks requiring speed or force a human can't apply by hand. It also logs exactly the state-action pairs the policy will eventually need. The tradeoff is that operating a robot remotely well is itself a skill, and demonstrations from a less-practiced teleoperator can be noisier or less efficient than kinesthetic teaching from someone with direct physical control.