Good at the simulator is not the same as good at reality
A policy trained entirely in simulation has effectively learned to be very good at the simulator, not necessarily at reality. Simulated physics engines use simplified friction models, approximate contact dynamics, and typically give the agent perfect, noise-free sensor readings with no manufacturing variation between robots. Real hardware has none of these conveniences, actual friction is messy and inconsistent, sensors are noisy and sometimes miscalibrated, and no two physical robots, even the same model off the same assembly line, behave in exactly identical ways.
This mismatch is called the sim-to-real gap, and it is the central practical challenge in applying RL to robotics. A policy that performs beautifully in simulation, smoothly grasping objects or walking without stumbling, can fail, sometimes badly, when deployed on real hardware, because it has implicitly learned to exploit exact regularities of the simulator that simply don't exist in the real world.