Neurosim: A Fast Simulator for Neuromorphic Robot Perception
2026-02-16 • Robotics
RoboticsComputer Vision and Pattern Recognition
AI summaryⓘ
The authors present Neurosim, a software tool that quickly simulates various sensors like cameras and motion sensors, as well as the movement of drones in changing environments. They also introduce Cortex, a communication system that helps Neurosim work smoothly with machine learning and robotics programs using Python and C++. Together, these tools allow researchers to train and test smart perception and control systems in real-time. The authors explain how their design supports learning from multiple types of synced sensor data and running experiments that respond instantly to what the sensors detect.
Dynamic Vision SensorNeuromorphic PerceptionMulti-rotor VehicleReal-time SimulationZeroMQMachine LearningSelf-supervised LearningPyTorchNumPyClosed-loop Control
Authors
Richeek Das, Pratik Chaudhari
Abstract
Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .