LDM:在潜空间里做高分辨率图像合成High-Resolution Image Synthesis with Latent Diffusion Models
问题:像素空间扩散效果好人尽皆知,但训练要几百 GPU 天、推理要反复过全分辨率网络;能不能先把图像压到潜空间,再在潜空间扩散,以此换效率?
证据位置:方法在 §3,包含感知压缩与语义压缩的拆分(§3.1)、潜空间扩散(§3.2)、cross-attention 条件机制(§3.3);实验在 §4(无条件、布局/类别/文本条件、超分、inpainting);局限在 §5 Limitations,结论在 §6。
作者自陈局限:§5 列了两条——第一,潜空间扩散仍是顺序采样,比 GAN 慢;第二,f=4 自编码器虽然画质损失很小,但重建能力会成为「像素级精细精度」任务的瓶颈,作者点名超分模型已受此限制。
它没主张的:没有说潜空间压缩无损——§5 明确说高精度场景可能不够;没有说 LDM 在所有任务上都超过像素扩散——摘要用「competitive」描述文本到图像、无条件生成与超分;也没有说 cross-attention 能理解任意条件——实验覆盖类别、文本、布局,但未证明开放域任意控制。Problem: pixel-space diffusion is high quality but costs hundreds of GPU days to train and repeatedly runs a full-resolution network at inference; can images be compressed into a latent space and diffused there to buy efficiency?
Where the evidence sits: method in Section 3, splitting perceptual from semantic compression (3.1), latent diffusion (3.2), and cross-attention conditioning (3.3); experiments in Section 4 (unconditional, layout/class/text conditioning, super-resolution, inpainting); limitations in Section 5 and conclusion in Section 6.
Stated limits: Section 5 lists two — latent diffusion still samples sequentially and is slower than GANs; and although the f=4 autoencoder loses very little quality, its reconstruction can bottleneck tasks needing fine-grained pixel accuracy, with super-resolution called out as already limited.
What it does not claim: it does not claim latent compression is lossless — Section 5 says high-precision settings may not suffice; it does not claim LDMs beat pixel diffusion on every task — the abstract calls text-to-image, unconditional generation and super-resolution “competitive”; and it does not claim cross-attention understands arbitrary conditions, since experiments cover class, text and layout but not open-ended control.
关键参数Key specs
- 证据位置 / evidence
- §3 方法(3.1–3.3);§4 实验;§5 Limitations;§6 结论
- 规模 / scale
- 公开的 latent diffusion 与 autoencoder 权重;最高约 1024² 像素
- 关键数字 / key numbers
- inpainting 与 class-conditional 达新 SOTA;文本到图像、无条件、超分为有竞争力
- 自陈局限 / stated limits
- 顺序采样慢于 GAN;f=4 自编码器重建限制像素级精细任务(§5)
- 复现状态 / reproduction
- 未复现
官方原文怎么说What the source says
we apply them in the latent space of powerful pretrained autoencoders
Our latent diffusion models (LDMs) achieve new state-of-the-art scores for image inpainting and class-conditional image synthesis
their sequential sampling process is still slower than that of GANs
资料与许可Source and licence
- 资料时间Source checked
- 2026-11-17
- 许可 / 类型Licence / kind
- arxiv-preprint
- 许可说明Licence evidence
- https://arxiv.org/abs/2112.10752
- 类目Category
- 架构Architecture
论文里的数字怎么读How to read the numbers
先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards