跳到主要内容Skip to main content

← 返回AI 论文← Back to AI Papers

架构Architecture

ViT:一张图等于 16×16 个词An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

问题:Transformer 在 NLP 已经成为标准,但在视觉里通常还要和 CNN 绑定;能不能把图像切成 patch、当成词序列,直接用纯 Transformer 做分类?
证据位置:方法在 §3(patch 切分、线性投影、位置编码);实验在 §4,包含不同预训练数据规模(ImageNet、ImageNet-21k、JFT-300M)与 ResNet/hybrid 对比;结论在 §5。
作者自陈局限:§5 明确列出三项——还没把 ViT 用到检测与分割;自监督预训练有提升,但与大规模监督预训练仍有很大差距;继续扩大规模可能还会更好,但未验证。§4 也承认在小数据集上 ViT 比同算力 ResNet 更容易过拟合,卷积的归纳偏置在数据少时仍然有用。
它没主张的:没有说 ViT 在所有视觉任务上取代 CNN——结论把检测/分割列为未来工作;没有说自监督已经追平监督;也没有说 patch 16×16 是最优切法(做了 patch size 与位置编码消融,但结论是「规模优先于归纳偏置」这一范围内)。
Problem: Transformers were already standard in NLP but usually tied to CNNs in vision; can an image be split into patches, treated as a token sequence, and classified by a pure Transformer?
Where the evidence sits: method in Section 3 (patch extraction, linear projection, positional encoding); experiments in Section 4 across pretraining data scales (ImageNet, ImageNet-21k, JFT-300M) and against ResNets and hybrids; conclusion in Section 5.
Stated limits: Section 5 lists three — ViT has not yet been applied to detection and segmentation; self-supervised pretraining helps but still lags large-scale supervised pretraining by a large gap; further scaling would likely help but was not verified. Section 4 also concedes that on small datasets ViT overfits more than a ResNet of comparable compute, so convolutional inductive bias still matters when data is scarce.
What it does not claim: it does not claim ViT replaces CNNs across all vision tasks — detection and segmentation are left to future work; it does not claim self-supervision has caught up with supervision; and it does not claim 16x16 is the optimal patch size beyond the ablations it ran.

关键参数Key specs

证据位置 / evidence
§3 方法;§4 实验与数据规模消融;§5 结论与局限
规模 / scale
ViT-Base / Large / Huge,patch 16 与 32 等变体
关键数字 / key numbers
ImageNet 88.55%、ReaL 90.72%、CIFAR-100 94.55%、VTAB 19 任务 77.63%
自陈局限 / stated limits
检测/分割未验证;自监督仍落后;小数据易过拟合(§4、§5)
复现状态 / reproduction
未复现

官方原文怎么说What the source says

the best model reaches the accuracy of 88.55% on ImageNet, 90.72% on ImageNet-ReaL, 94.55% on CIFAR-100, and 77.63% on the VTAB suite of 19 tasks
1 https://ar5iv.labs.arxiv.org/html/2010.11929
One is to apply ViT to other computer vision tasks, such as detection and segmentation.
2 https://ar5iv.labs.arxiv.org/html/2010.11929
Our initial experiments show improvement from self-supervised pre-training, but there is still large gap between self-supervised and large-scale supervised pre-training.
3 https://ar5iv.labs.arxiv.org/html/2010.11929

资料与许可Source and licence

资料时间Source checked
2026-11-17
许可 / 类型Licence / kind
arxiv-preprint
许可说明Licence evidence
https://arxiv.org/abs/2010.11929
类目Category
架构Architecture

论文里的数字怎么读How to read the numbers

先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards

去官网看原页Open the official page →

← 返回AI 论文← Back to AI Papers