AWQ: Activation-aware Weight Quantization for LLM Compression and AccelerationAWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
问题:LLM 低比特权重量化如何保精度、又保持硬件友好,能落到端侧。
证据位置:摘要称只保护 1% 显著权重即可大幅降低量化误差;显著通道要按激活分布而非权重判断;用等价缩放保护显著通道;不依赖反向传播或重建;TinyChat 在桌面/移动 GPU 上相对 HF FP16 有 3× 以上加速;可在移动 GPU 跑 70B Llama-2。
作者自陈局限:权重量化,激活仍较高精度;校准集有偏;3× 是特定实现/硬件;未覆盖所有模型结构。
它没主张的:不是训练方法;不是 4-bit 无损;不是所有设备都能跑 70B。Problem: how to quantise LLM weights to low bits while keeping accuracy and hardware-friendly deployment on edge devices?
Where the evidence sits: the abstract says protecting only 1% salient weights greatly reduces quantisation error; saliency should be judged from activations, not weights; an equivalent scaling protects salient channels; no backprop or reconstruction is used; TinyChat reports >3x speedup over HF FP16 on desktop and mobile GPUs and runs 70B Llama-2 on mobile GPUs.
What the authors say about limits: weight-only quantisation, activations stay higher precision; calibration set bias; 3x is implementation/hardware-specific; not every model architecture is covered.
What it does not claim: not a training method; not lossless 4-bit; not every device can run 70B.
关键参数Key specs
- 证据位置 / evidence
- 摘要:1% 显著权重、激活分布、3× 加速
- 权重量化 / weight bits
- 4-bit
- 复现状态 / reproduction
- 未复现(只读摘要)
官方原文怎么说What the source says
Protecting only 1% salient weights can greatly reduce quantization error
we should refer to the activation distribution, not weights
more than 3x speedup over the Huggingface FP16 implementation
资料与许可Source and licence
- 资料时间Source checked
- 2026-11-19
- 许可 / 类型Licence / kind
- arxiv-preprint
- 许可说明Licence evidence
- https://arxiv.org/abs/2306.00978
- 类目Category
- 效率Efficiency
论文里的数字怎么读How to read the numbers
先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards