Human-to-Robot Retargeting : Putting Pinocchio to the Test
Human-to-Robot Retargeting : Putting Pinocchio to the Test
Pinocchio, the foundational building block of the MAESTRO software stack, is being put to the test in a real-world use case: retargeting human movements to a humanoid robot.
The Project
Ajay Sathya, a researcher at Inria on the WILLOW team (CNRS / ENS-PSL/ Inria), along with his two interns, Guillaume Besset and Erwann Carn, are working on transferring human movements to a Unitree G1 humanoid robot. The challenge of retargeting: mapping movement captured from a human body to a very different robotic kinematics (number of joints, ranges of motion, proportions, mechanical constraints), while preserving the intent and dynamics of the original gesture.
Two movements serve as test cases, chosen for their distinct dynamic profiles:
Walking : a functional, cyclic movement, dominated by alternating foot contacts and balance transfers on the ground.
The famous Macarena (WITH MUSIC !!) : a rhythmic, expressive movement, which primarily engages the upper body with wide joint ranges of motion and few ground-contact constraints;
This diversity allows us to evaluate the retargeting pipeline’s ability to generalize beyond a single type of gesture, covering both contact and balance constraints as well as pure joint-amplitude constraints.
Pinocchio’s Role
Pinocchio is used to calculate the correspondence between the joints of the captured human body and those of the G1; it is what enables the mathematical translation of a movement from one morphology to another. This retargeting component is an original contribution by the team: it relies entirely on Pinocchio for the kinematic modeling required for this mapping between feet, hips, and shoulders. The result of the retargeting is then used to train a reinforcement learning policy, before being executed on the physical robot.
In parallel: simulation and real robot
The movements were reenacted both in simulation in MuJoCo and on Inria’s robotics platform in Paris, using the physical G1 robot.
Links
Pinocchio GitHub repository