Segmentation, pose estimation, then planning
Turning a camera or depth sensor reading into an executed grasp is a multi-stage pipeline, and each stage depends on the one before it working correctly. The first stage is object detection and segmentation, which isolates the target object from the background and from other objects around it, drawing a precise boundary around exactly what should be picked. The second stage is pose estimation, which figures out the object's orientation and position in 3D space, not just where it is in the 2D image but which way it's facing and how it's tilted. Only after both of those succeed does the grasp planner from the previous lesson get involved, taking the segmented shape and estimated pose and searching for the best contact points.
This staged structure mirrors the sense-plan-act pattern from general robot perception, but manipulation adds a layer of precision that navigation doesn't need. A self-driving robot can tolerate a few centimeters of position error and just adjust course. A gripper closing on the wrong point by a few centimeters often means missing the object completely or knocking it out of position before contact is even made.