AI summaryⓘ
The authors explain a connection between how diffusion models add noise to images and how traditional scale-space theory simplifies images by blurring. They show that very noisy images in diffusion models contain about the same information as small, low-resolution images, questioning why full resolution is always used. To improve this, they create a new method called Scale Space Diffusion that combines different levels of image detail during the denoising process. They also design a special neural network, Flexi-UNet, which efficiently handles images at multiple resolutions. Their tests on popular image datasets demonstrate how their approach works across different image sizes and network complexities.
Diffusion modelsNoise degradationScale-space theoryLow-pass filteringDownsamplingDenoisingUNetResolution scalingImageNetCelebA
Authors
Soumik Mukhopadhyay, Prateksha Udhayanan, Abhinav Shrivastava
Abstract
Diffusion models degrade images through noise, and reversing this process reveals an information hierarchy across timesteps. Scale-space theory exhibits a similar hierarchy via low-pass filtering. We formalize this connection and show that highly noisy diffusion states contain no more information than small, downsampled images - raising the question of why they must be processed at full resolution. To address this, we fuse scale spaces into the diffusion process by formulating a family of diffusion models with generalized linear degradations and practical implementations. Using downsampling as the degradation yields our proposed Scale Space Diffusion. To support Scale Space Diffusion, we introduce Flexi-UNet, a UNet variant that performs resolution-preserving and resolution-increasing denoising using only the necessary parts of the network. We evaluate our framework on CelebA and ImageNet and analyze its scaling behavior across resolutions and network depths. Our project website ( https://prateksha.github.io/projects/scale-space-diffusion/ ) is available publicly.