TerraZero: Scaling RL for Autonomous Driving

The quest for robust autonomous driving agents hinges on simulators that can rapidly generate diverse, safety-critical scenarios. Existing solutions often trade speed for realism or fail to capture the long tail of edge cases present in real-world driving.

Unlocking Scalable Reinforcement Learning with Extreme Throughput

The newly introduced TerraZero autonomous driving simulator redefines the performance envelope for training autonomous agents. By leveraging a C engine for simulation on the CPU and policy inference on the GPU via a zero-copy path, TerraZero sustains an impressive 1.3 million agent-steps per second on a single server-grade GPU. This throughput dramatically outpaces traditional object-level simulators, enabling reinforcement learning at an unprecedented scale.

Procedural Generation for an Unbounded Scenario Space

TerraZero tackles the critical challenge of covering the safety-critical long tail of driving scenarios. Instead of relying solely on logged data, which is inherently limited, the simulator populates real-world map geometries with randomized rule-based road users and signal controllers. Furthermore, agent dynamics, rewards, and sizes are randomized per episode. This procedural approach ensures that each map can yield an unbounded set of diverse scenarios, moving beyond the limitations of static datasets.

From Scratch to Benchmark Dominance: Zero-Shot Generalization

A key outcome of the TerraZero framework is its ability to train fully learned policies from scratch using reinforcement learning alone, without human demonstrations or fallback planners at inference. These policies demonstrate remarkable zero-shot generalization capabilities, successfully navigating across different cities and datasets, even learning emergent behaviors like left-hand-traffic driving without explicit supervision. Notably, a TerraZero-trained policy achieved top performance on the InterPlan long-tail benchmark and showcased superior safety metrics on routine driving tasks, outperforming larger learned planners and demonstrating competitive results against reference-anchored self-play methods on the Waymo Open Sim Agents benchmark.