StandardE2E: A Unified Framework for End-to-End Autonomous Driving Datasets
2026-06-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors noticed that self-driving car research uses many different datasets, each with its own format and rules, making it hard to work with several datasets at once. They created StandardE2E, a tool that brings these datasets into one common format so researchers can use them together easily. This tool also helps prepare data in the same way and supports training models across multiple datasets without changing the rest of the process. They provide support for six popular driving datasets and shared their work as an open-source Python package.
autonomous drivingend-to-end modelssensor datasetsdata preprocessingPyTorch DataLoader3D detectionmotion forecastingHD mapscross-dataset trainingopen-source framework
Authors
Stepan Konev
Abstract
Autonomous driving has shifted from modular perception-prediction-planning stacks toward end-to-end (E2E) models that map sensor inputs directly to vehicle control, often regularized by auxiliary tasks such as 3D detection, motion forecasting, and HD-map perception. Progress is driven by a fast-growing ecosystem of sensor-rich driving datasets, yet each ships its own file formats, APIs, coordinate conventions, and modality coverage, leaving cross-dataset experimentation and even basic per-dataset preprocessing to be re-implemented per project. We present StandardE2E, a framework that provides a single unified interface over E2E driving datasets. StandardE2E (i) standardizes per-dataset preprocessing under one shared data schema; (ii) combines multiple datasets in a single PyTorch DataLoader for cross-dataset pretraining, auxiliary-task supervision, and scenario-level filtering; and (iii) reduces adding a new dataset to a single per-dataset mapping from raw frames to the canonical schema, leaving the entire downstream pipeline unchanged. The framework supports six datasets out of the box: Waymo End-to-End, Waymo Perception, Argoverse 2 Sensor, Argoverse 2 LiDAR, NAVSIM (OpenScene-v1.1), and WayveScenes101, and is released as the open-source standard-e2e Python package, available at https://github.com/stepankonev/StandardE2E.