Same points, through time
Persistent point correspondence ties the robot and objects to their future trajectories.
Motion for action · World Action Models
Trace-WAMLearning to Act via
3D Motion Prediction
for World Action Models
† Corresponding authors
Learning how the robot and objects move.
Turning future 3D motion into better actions.
Predicting motion
Across space and time.
Learning to act
to real-world manipulation
98.7%LIBERO
84.5%LIBERO-Plus
80.3%RoboCasa-GR1 · Full data
85.0%Real world · Average
01 / The idea
The future is more
than appearance.
It is motion.
World Action Models couple future visual prediction with action generation. But successful manipulation depends on how the robot and task-relevant objects move—not just how the next frame looks.
Trace-WAM adds future 3D motion prediction as an auxiliary objective. Tracking the same points through time captures both image-plane displacement and depth change, providing a direct learning signal for interaction dynamics.
Learn with future motion.
Act from current observations.
02 / The method
Three experts.
One efficient policy.
Joint training enriches the representations used for action generation.
Tri-MoT architecture. Three Transformer experts jointly learn video prediction, 3D motion prediction, and action generation. At inference, action denoising uses cached visual features; future-video and motion prediction are removed.
View full resolutionBeyond the image plane
Image displacement and depth change describe motion in a unified reference-camera frame.
Auxiliary at training time
Deployment uses a single visual forward pass and action denoising, without future motion generation.
03 / Real-world manipulation
Motion learned.
Actions grounded.
Three tasks on a Piper robot, from precise placement to sustained interaction.
85.0%average success rate
Real world
Place Bread
85%Place the bread in the bowl.
Real world
Place and Close Drawer
95%Place the object inside, then close the drawer.
Real world
Wipe Blackboard
75%Erase the chalk markings from the board.
Representative successful executions · 20 evaluation trials per task and method
+21.7 percentage points in average success over Fast-WAM. Explore the complete results ↓
04 / Simulation
Across tasks.
Across environments.
From diverse manipulation tasks to seven dimensions of distribution shift.
Place the can and close the drawer
RoboCasa-GR1
Place the bottle and close the cabinet
RoboCasa-GR1
Transfer an object from tray to pot
RoboCasa-GR1
Representative successful rollouts · Click a video to play
05 / Benchmarks
Stronger action learning.
Success rates across three simulation benchmarks.
06 / The paperLearning motion.
THE PAPER
Learning motion.
Improving action.
Explore the full method, experimental results, and the details behind motion-auxiliary learning.
Read the paper
Trace-WAM
Learning to Act via 3D Motion Prediction for World Action Models
Citation