BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsBLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
问题:视觉语言预训练成本高,能否冻结图像编码器与大语言模型,只训练轻量桥接模块。
证据位置:摘要给出冻结图像编码器与冻结 LLM,用轻量 Querying Transformer 两阶段预训练;在 zero-shot VQAv2 上比 Flamingo80B 高 8.7%,可训练参数少 54×;支持零样本图生文指令。
作者自陈局限:冻结组件限制领域适配;两阶段训练仍有数据依赖;评测以 VQA/生成任务为主;未证明所有模态与语言。
它没主张的:不是无训练;不是取代全量微调;不是所有任务都优于大模型。Problem: vision-language pretraining is expensive; can we freeze the image encoder and the LLM and train only a lightweight bridge?
Where the evidence sits: the abstract uses frozen pretrained image encoders and frozen LLMs with a lightweight Querying Transformer trained in two stages; it reports +8.7% on zero-shot VQAv2 over Flamingo80B with 54x fewer trainable parameters and supports zero-shot image-to-text instructions.
What the authors say about limits: frozen components limit domain adaptation; two-stage training still depends on data; evaluation centres on VQA/generation; not all modalities or languages are covered.
What it does not claim: not training-free; not a replacement for full fine-tuning; not better than large models on every task.
关键参数Key specs
- 证据位置 / evidence
- 摘要:冻结编码器/LLM、54× 参数、+8.7%
- 可训练参数 / trainable
- 54× fewer than Flamingo80B
- 复现状态 / reproduction
- 未复现(只读摘要)
官方原文怎么说What the source says
bootstraps vision-language pre-training from off-the-shelf frozen pre-trained image encoders and frozen large language models
bridges the modality gap with a lightweight Querying Transformer
outperforms Flamingo80B by 8.7% on zero-shot VQAv2 with 54x fewer trainable parameters
资料与许可Source and licence
- 资料时间Source checked
- 2026-11-19
- 许可 / 类型Licence / kind
- arxiv-preprint
- 许可说明Licence evidence
- https://arxiv.org/abs/2301.12597
- 类目Category
- 多模态Multimodal
论文里的数字怎么读How to read the numbers
先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards