跳到主要内容Skip to main content

← 返回AI 论文← Back to AI Papers

对齐Alignment

Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RLAbstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL

问题:模型要么硬答、要么一句「无法回答」结束;缺的是「该弃答时弃答,弃答后还能问清缺什么」的可校准行为。
证据位置:摘要报告一个 3B 模型,用 clarification-aware RLVR 奖励,在不可答查询上同时改善弃答与澄清,并保持可答查询上的表现;方法属于可验证奖励的强化学习,不是提示词模板。
作者自陈局限:3B 规模,不能直接外推到 70B+;训练/评测任务以不可答查询为主,开放域闲聊与多轮澄清覆盖有限;奖励设计依赖可验证的答案/澄清标签,标签本身有噪声;没有生产部署验证。
它没主张的:不是通用助手模型;不是解决所有幻觉;不是「弃答越多越好」;不是不需要人工评估的自动对齐方案。
Problem: models either answer anyway or stop with “cannot answer”; what is missing is calibratable behaviour that abstains when appropriate and then asks for the missing information.
Where the evidence sits: the abstract reports a 3B model trained with a clarification-aware RLVR reward that improves abstention and clarification on unanswerable queries while preserving strong performance on answerable ones; it is verifiable-reward RL, not a prompt template.
What the authors say about limits: 3B scale, not directly extrapolable to 70B+; training and evaluation centre on unanswerable queries, with limited coverage of open-domain chat and multi-turn clarification; the reward depends on verifiable answer/clarification labels, which carry noise; no production deployment is reported.
What it does not claim: a general assistant model; a solution to all hallucinations; that more abstention is always better; an auto-alignment method that needs no human evaluation.

关键参数Key specs

发表 / venue
arXiv 2604.17073;ACL 2026
模型规模 / scale
3B
方法 / method
clarification-aware RLVR
目标 / target
可校准弃答 + 拒答后澄清
证据位置 / evidence
摘要;方法段;实验段
复现状态 / reproduction
未复现

官方原文怎么说What the source says

a 3B model that improves abstention and clarification on unanswerable queries while preserving strong performance on answerable ones
1 https://arxiv.org/abs/2604.17073
clarification-aware RLVR reward
2 https://arxiv.org/abs/2604.17073

资料与许可Source and licence

资料时间Source checked
2026-11-18
许可 / 类型Licence / kind
arxiv-preprint
许可说明Licence evidence
https://arxiv.org/abs/2604.17073
类目Category
对齐Alignment

论文里的数字怎么读How to read the numbers

先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards

去官网看原页Open the official page →

← 返回AI 论文← Back to AI Papers