Searching for a feasible path, not just a target
For a fully-actuated system, getting from one state to another is often straightforward: command each degree of freedom along a reasonable path and let the controller track it. For an underactuated system, not every path between two states is achievable, because the unactuated degrees of freedom can only move in ways consistent with the system's dynamics. Trajectory optimization addresses this by searching for a sequence of states and actuator inputs, over time, that both respects those dynamics and drives the system from a start state toward a goal, often while minimizing some cost such as control effort or time.
This turns the control problem into a numerical optimization problem: given the equations of motion, the actuator limits, and a start and goal, find the sequence of inputs that connects them. The output isn't just a single command but an entire planned motion, which a feedback controller can then track and correct against, compensating for small disturbances or modeling errors along the way.