WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory
2026-07-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce WorldDirector, a new system that creates videos where objects move realistically and keep their appearance even after disappearing and coming back. Unlike older models that mix how things move with how they look, this system separates the two, using a language model to plan object and camera movements first. Then, it uses these plans to generate videos that follow real-world logic and keep things consistent. Their experiments show it can handle complex scenes with better control and memory of moving objects over time.
video world modeldynamic object memoryviewpoint explorationsemantic motion orchestrationvisual generationLLM (Large Language Model)3D trajectoriesvideo synthesisappearance stabilityphysical dynamics
Authors
Hanlin Wang, Hao Ouyang, Qiuyu Wang, Wen Wang, Qingyan Bai, Ka Leong Cheng, Yue Yu, Yixuan Li, Yihao Meng, Zichen Liu, Yanhong Zeng, Yujun Shen, Qifeng Chen
Abstract
We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/