Helix4D: Complex 4D Mesh Generation
2026-05-25 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce Helix4D, a new method for turning videos into detailed 4D meshes that can handle tricky things like transparent objects and thin parts better than before. They improved an earlier method, Trellis2, by making it share information across different video frames without losing quality on hard cases. They also added a way to include time information into the 3D encoding smoothly, without needing extra parameters. Their tests show Helix4D works well on standard and complex dynamic scenes.
video-to-4Ddynamic mesh generationTrellis2frame-local attentioncross-frame attentiontemporal encoding3D positional encodingtransparent materialsthin structuresRoPE
Authors
Jiraphon Yenphraphai, Jianqi Chen, Jian Wang, Gordon Qian, Sergey Tulyakov, Rameen Abdal, Raymond A. Yeh, Peter Wonka, Chaoyang Wang
Abstract
Current video-to-4D methods struggle with complex topology changes, transparent materials, thin structures, and inner surfaces. We present Helix4D, a dynamic mesh generation framework by inheriting the expressive representation of Trellis2, adapting it from image-to-3D to video-conditioned 4D generation. Our design arises from two key questions: (a) how to enable Trellis2's frame-local attention to share information across frames while preserving its pretrained quality on rare cases such as transparent objects and inner surfaces, and (b) how to inject temporal information into a purely 3D positional encoding without breaking pretrained capabilities. We address (a) with a sliding-window cross-frame attention and anchor on the first frame. The first frame is generated by the base Trellis2 model and injected into our model, letting it inherit Trellis2's quality in rare cases through cross-frame attention. We address (b) with a 4D temporal encoding that repurposes redundant low-frequency spatial RoPE bands for time, extending the encoding from 3D with no additional parameters. Extensive experiments show the effectiveness of Helix4D for high-quality dynamic mesh generation on ActionBench and our own challenging complex dynamics set.