Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RLAbstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL
问题:模型要么硬答、要么一句「无法回答」结束;缺的是「该弃答时弃答,弃答后还能问清缺什么」的可校准行为。
证据位置:摘要报告一个 3B 模型,用 clarification-aware RLVR 奖励,在不可答查询上同时改善弃答与澄清,并保持可答查询上的表现;方法属于可验证奖励的强化学习,不是提示词模板。
作者自陈局限:3B 规模,不能直接外推到 70B+;训练/评测任务以不可答查询为主,开放域闲聊与多轮澄清覆盖有限;奖励设计依赖可验证的答案/澄清标签,标签本身有噪声;没有生产部署验证。
它没主张的:不是通用助手模型;不是解决所有幻觉;不是「弃答越多越好」;不是不需要人工评估的自动对齐方案。Problem: models either answer anyway or stop with “cannot answer”; what is missing is calibratable behaviour that abstains when appropriate and then asks for the missing information.
Where the evidence sits: the abstract reports a 3B model trained with a clarification-aware RLVR reward that improves abstention and clarification on unanswerable queries while preserving strong performance on answerable ones; it is verifiable-reward RL, not a prompt template.
What the authors say about limits: 3B scale, not directly extrapolable to 70B+; training and evaluation centre on unanswerable queries, with limited coverage of open-domain chat and multi-turn clarification; the reward depends on verifiable answer/clarification labels, which carry noise; no production deployment is reported.
What it does not claim: a general assistant model; a solution to all hallucinations; that more abstention is always better; an auto-alignment method that needs no human evaluation.
关键参数Key specs
- 发表 / venue
- arXiv 2604.17073;ACL 2026
- 模型规模 / scale
- 3B
- 方法 / method
- clarification-aware RLVR
- 目标 / target
- 可校准弃答 + 拒答后澄清
- 证据位置 / evidence
- 摘要;方法段;实验段
- 复现状态 / reproduction
- 未复现
官方原文怎么说What the source says
a 3B model that improves abstention and clarification on unanswerable queries while preserving strong performance on answerable ones
clarification-aware RLVR reward
资料与许可Source and licence
- 资料时间Source checked
- 2026-11-18
- 许可 / 类型Licence / kind
- arxiv-preprint
- 许可说明Licence evidence
- https://arxiv.org/abs/2604.17073
- 类目Category
- 对齐Alignment
论文里的数字怎么读How to read the numbers
先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards