A Pytorch-Enabled Tool for Synthetic Event Camera Data Generation and Algorithm Development

Event, or neuromorphic cameras, offer a novel encoding of natural scenes by asynchronously reporting changes in brightness as binary “event” indicators with improved dynamic range, temporal resolution, and data bandwidth when compared to conventional cameras. However, their adoption is slowed in part by their limited commercial availability, lack of existing datasets, and challenges related to predicting their nonlinear optical encoding, unique noise models, and tensor-based data processing requirements in domain-specific tasks. To address these challenges, we introduce Synthetic Events for Neural Processing and Integration (SENPI) in Python, a PyTorch-enabled library for simulating and processing events. SENPI includes a differentiable digital twin that converts intensity-based data into event-based representations to evaluate event camera performance and augment existing data using a physics-driven forward model. The library also supports modules for event-based I/O, manipulation, filtering, and visualization, creating efficient and scalable workflows for both synthetic and real events. Using SENPI’s physical modeling capabilities, we analyze the tradeoff in false alarm vs true events under photometric and sensor noise sources to propose a method to optimize event contrast thresholds for deployment under natural conditions. Next, we demonstrate SENPI’s ability to produce realistic events by comparing the synthetic outputs to real events and use these results to draw conclusions on the properties of event-based perception. Ultimately, SENPI lowers the barrier to entry for researchers by providing an accessible tool for event data generation and algorithmic development, making it a valuable resource for advancing research in neuromorphic vision systems.