RoboTTT: Context Scaling for Robot Policies
2026-07-16 • Robotics
RoboticsArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors present RoboTTT, a new robot model that can learn from very long sequences of visual and motor data—much longer than previous models—without slowing down. This allows robots to imitate actions from a single human video, improve themselves while working, and handle more complex tasks reliably. Their approach uses a special training method that updates the robot’s internal settings during both learning and real use, helping it remember and use long past information. Experiments show RoboTTT greatly outperforms earlier models, completing long multi-step tasks that others could not. The authors also found that increasing the length of experience the model trains on steadily improves its performance.
robot foundation modelsvisuomotor contextTest-Time Trainingone-shot imitationsequence modelstruncated backpropagation through timelong-horizon tasksclosed-loop performancefast weightspretraining
Authors
Yunfan Jiang, Yevgen Chebotar, Ruijie Zheng, Fengyuan Hu, Yunhao Ge, Jimmy Wu, Tianyuan Dai, Scott Reed, Li Fei-Fei, Yuke Zhu, Linxi "Jim" Fan
Abstract
Recent robot foundation models operate with single-step or short-history visuomotor context. We introduce Test-Time-Training Robot Policies (RoboTTT), a robot model and training recipe that scale visuomotor context to 8K timesteps, three orders of magnitude beyond state-of-the-art policies, without growing inference latency. At this context length, we unlock new robot capabilities: one-shot in-context imitation from human video demonstrations, on-the-fly policy improvement, robustness to perturbations, and stronger performance on multi-stage, long-horizon tasks. We also observe, for the first time, steady gains in closed-loop performance as pretraining context length scales. At its core, RoboTTT integrates Test-Time Training into robot foundation models such as Vision-Language-Action policies, yielding a sequence model whose recurrent state consists of fast weights, parameters updated by gradient descent during both training and inference, compressing histories into weight space and retrieving contextual information for long-context conditioning. To scale training context length, the recipe combines sequence action forcing with truncated backpropagation through time. On challenging real-robot manipulation tasks, RoboTTT improves overall performance by 87% over the single-step context baseline and fully completes a five-minute, ten-stage assembly task, which no baseline ever does. RoboTTT trained with 8K-timestep context outperforms the same model pretrained with 1K timesteps by 62%, suggesting context length as a new scaling axis for robot foundation models. Videos are available at https://research.nvidia.com/labs/gear/robottt/