AI summaryⓘ
The authors developed Lumosaic, a small device that captures color details beyond normal video at many wavelengths, and does this quickly for moving scenes. It uses special LEDs and a camera that can control exposure for each pixel very fast, combining light patterns and timing to better capture colors even when things move. Their method improves on older systems by actively controlling lighting and exposure, which helps keep colors accurate and images stable over time. They use machine learning to turn the captured data into detailed videos with 31 color channels, working at normal video speeds and resolution. Tests show their system produces clearer and more consistent color videos than existing snapshot methods.
hyperspectral imagingcoded exposureLED illuminationpixel-wise exposure controlsnapshot imagingspectral fidelitymachine learning reconstructiondynamic scenesvideo frame encodingphoton utilization
Authors
Dhruv Verma, Andrew Qiu, Roberto Rangel, Ayandev Barman, Hao Yang, Chenjia Hu, Fengqi Zhang, Roman Genov, David B. Lindell, Kiriakos N. Kutulakos, Alex Mariakakis
Abstract
We present Lumosaic, a compact active hyperspectral video system designed for real-time capture of dynamic scenes. Our approach combines a narrowband LED array with a coded-exposure-pixel (CEP) camera capable of high-speed, per-pixel exposure control, enabling joint encoding of scene information across space, time, and wavelength within each video frame. Unlike passive snapshot systems that divide light across multiple spectral channels simultaneously and assume no motion during a frame's exposure, Lumosaic actively synchronizes illumination and pixel-wise exposure, improving photon utilization and preserving spectral fidelity under motion. A learning-based reconstruction pipeline then recovers 31-channel hyperspectral (400-700 nm) video at 30 fps and VGA resolution, producing temporally coherent and spectrally accurate reconstructions. Experiments on synthetic and real data demonstrate that Lumosaic significantly improves reconstruction fidelity and temporal stability over existing snapshot hyperspectral imaging systems, enabling robust hyperspectral video across diverse materials and motion conditions.