跳到主要内容Skip to main content

← 返回AI 论文← Back to AI Papers

效率Efficiency

AWQ: Activation-aware Weight Quantization for LLM Compression and AccelerationAWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

问题:LLM 低比特权重量化如何保精度、又保持硬件友好,能落到端侧。
证据位置:摘要称只保护 1% 显著权重即可大幅降低量化误差;显著通道要按激活分布而非权重判断;用等价缩放保护显著通道;不依赖反向传播或重建;TinyChat 在桌面/移动 GPU 上相对 HF FP16 有 3× 以上加速;可在移动 GPU 跑 70B Llama-2。
作者自陈局限:权重量化,激活仍较高精度;校准集有偏;3× 是特定实现/硬件;未覆盖所有模型结构。
它没主张的:不是训练方法;不是 4-bit 无损;不是所有设备都能跑 70B。
Problem: how to quantise LLM weights to low bits while keeping accuracy and hardware-friendly deployment on edge devices?
Where the evidence sits: the abstract says protecting only 1% salient weights greatly reduces quantisation error; saliency should be judged from activations, not weights; an equivalent scaling protects salient channels; no backprop or reconstruction is used; TinyChat reports >3x speedup over HF FP16 on desktop and mobile GPUs and runs 70B Llama-2 on mobile GPUs.
What the authors say about limits: weight-only quantisation, activations stay higher precision; calibration set bias; 3x is implementation/hardware-specific; not every model architecture is covered.
What it does not claim: not a training method; not lossless 4-bit; not every device can run 70B.

关键参数Key specs

证据位置 / evidence
摘要:1% 显著权重、激活分布、3× 加速
权重量化 / weight bits
4-bit
复现状态 / reproduction
未复现(只读摘要)

官方原文怎么说What the source says

Protecting only 1% salient weights can greatly reduce quantization error
1 https://arxiv.org/abs/2306.00978
we should refer to the activation distribution, not weights
2 https://arxiv.org/abs/2306.00978
more than 3x speedup over the Huggingface FP16 implementation
3 https://arxiv.org/abs/2306.00978

资料与许可Source and licence

资料时间Source checked
2026-11-19
许可 / 类型Licence / kind
arxiv-preprint
许可说明Licence evidence
https://arxiv.org/abs/2306.00978
类目Category
效率Efficiency

论文里的数字怎么读How to read the numbers

先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards

去官网看原页Open the official page →

← 返回AI 论文← Back to AI Papers