SyncSpace: Layout-Conditioned 3D Gaussian Splatting for Space Reskinning in Mixed Reality
2026-07-11 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors developed SyncSpace, a system that helps create virtual 3D worlds that match the layout and style of a real physical space. They start by scanning the real space to get its shape, then use this data to guide a 3D generative model, which creates a stylized virtual scene. The system aligns this virtual scene with the real space using both automatic and manual adjustments. They demonstrate interactions where the virtual world can replace the physical one and show that the same space can be reimagined in different visual styles while keeping its shape.
3D generative modeldepth sensingspatial alignmentGaussian splat scenelayout panoramaregistration algorithmmanual refinementvirtual-physical interactionreskinninghand tracking
Authors
Qinchuan Zhang, Weibo Xu, Yunge Wen
Abstract
We present SyncSpace, a system that achieves both spatial alignment and visual consistency between a generated 3DGS world and physical space. We first scan the space via depth sensing to extract 3D bounding boxes, which we render into a layout-only panorama and feed as a geometric prior to a generative world model, producing a Gaussian splat scene in which objects are re-semantized to fit a target style without per-object control. We then align the generated scene to physical space with a coarse-to-fine registration algorithm, refined manually via pinch gestures when automatic registration does not converge. We demonstrate a hand-tracked engulfment interaction in which the virtual world rises to replace the physical space, and show a single space reskinned into multiple stylistically distinct worlds with its layout preserved.