TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale
2026-07-14 • Machine Learning
Machine LearningArtificial IntelligenceRobotics
AI summaryⓘ
The authors developed TerraZero, a fast and flexible driving simulator designed for training autonomous car software through reinforcement learning. It creates many varied driving scenarios by mixing real-world map data with randomized traffic and agent behaviors, all running efficiently on standard hardware. Their approach requires no human driving examples and trains driving policies that perform well across different cities and vehicles. These policies showed strong safety and realism on multiple benchmarks, including challenging rare cases. TerraZero also models various road users like pedestrians and cyclists in one unified system.
autonomous drivingreinforcement learningsimulatorself-playprocedural generationpolicy trainingmap geometrytraffic simulationego policytransfer learning
Authors
Zhouchonghao Wu, Akshay Rangesh, Weixin Li, Wei-Jer Chang, Zachary Lee, Tim Wang, Wei Zhan
Abstract
Training robust autonomous driving agents requires a simulator that is fast enough for reinforcement learning at scale, realistic enough to ground behavior in real-world map structure, and diverse enough to cover the safety-critical long tail that logged data rarely contains. We present TerraZero, a procedural driving simulator and self-play training stack. A configurable C engine runs simulation on the CPU and policy inference on the GPU over a zero-copy path, sustaining 1.3M agent-steps per second on a single server-grade GPU, far faster than existing object-level simulators, while keeping fidelity lighter single-agent systems omit: heterogeneous agents, multiple dynamics models, and full traffic-rule enforcement. TerraZero treats logged data only as a source of real-world map geometry, populating each map with randomized rule-based road users and signal controllers and randomizing agent dynamics, rewards, and sizes per episode, so a map yields an unbounded set of scenarios. Every reported policy trains from scratch by reinforcement learning alone on a compute-efficient self-play recipe across GPUs, with zero human demonstrations and no fallback planner at inference. Policies generalize zero-shot across cities and datasets, including emergent left-hand-traffic driving without explicit supervision. As an ego policy, TerraZero is the first fully learned policy to top the InterPlan long-tail benchmark, ahead of larger learned planners; on routine-driving val14 it ranks among the best approaches and is the safest, posting the best collision and time-to-collision scores. On Waymo Open Sim Agents realism the same recipe outperforms other demonstration-free methods and is competitive with the strongest reference-anchored self-play method. One stack serves both roles: driving policies across dynamics for cars and trucks, and sim agents that jointly control vehicles, pedestrians, and cyclists.