跳到主要内容Skip to main content

← 返回AI 论文← Back to AI Papers

架构Architecture

Proving and Improving the Platonic Representation Hypothesis with JEPAsProving and Improving the Platonic Representation Hypothesis with JEPAs

问题:不同模态的模型是否在收敛到同一个「柏拉图式」的表征空间?如果会,小模型能否用更好的训练视角追上大模型?
证据位置:摘要报告一个 109M 参数的文本 JEPA,在跨模态对齐上优于作者评测过的所有语言模型(最大到 72B);同段给出「好视角比 next-token 少 42 倍样本」的结论。正文用 JEPA 作为可证明/可改进的实例,而不是只做相关性观察。
作者自陈局限:这是预印本;文本 JEPA 只有 109M,规模远小于被比较的 LLM;对齐指标衡量的是表征相似度,不是生成质量;42× 是特定视角与数据条件下的样本效率,不是通用常数;没有证明所有模态都收敛到同一空间。
它没主张的:不是新的 SOTA 语言模型;不是 next-token prediction 的替代品;不是「小模型全面超过大模型」;不是对 PRH 的最终证明。
Problem: are models across modalities converging on a shared “Platonic” representation space? If so, can a small model catch up with a large one by using better training views?
Where the evidence sits: the abstract reports a 109M-parameter text JEPA that aligns with DINOv2 and CLIP image encoders better than every language model the authors evaluated, up to 72B; the same abstract states that good views need 42 times fewer training samples than next-token prediction. The body uses JEPA as a provable/improveable instance rather than a correlation-only observation.
What the authors say about limits: it is a preprint; the text JEPA is 109M, far smaller than the LLMs it is compared with; the alignment metric measures representation similarity, not generation quality; 42× is sample efficiency under specific views and data, not a universal constant; it does not prove every modality converges to one space.
What it does not claim: a new state-of-the-art language model; a replacement for next-token prediction; that small models beat large models across the board; a final proof of the Platonic Representation Hypothesis.

关键参数Key specs

发表 / venue
arXiv 2610.39302,2026-10-29
作者 / authors
Balestriero & LeCun
模型 / model
109M 参数文本 JEPA
关键数字 / key
42× 更少样本;对齐优于最大 72B 的 LLM
证据位置 / evidence
摘要;JEPA 实验段
复现状态 / reproduction
未复现

官方原文怎么说What the source says

a 109M-parameter text JEPA aligns with DINOv2 and CLIP's image encoders better than every language model we evaluated, up to 72B parameters
1 https://arxiv.org/abs/2610.39302
Good views need 42 times fewer training samples than next-token prediction
2 https://arxiv.org/abs/2610.39302

资料与许可Source and licence

资料时间Source checked
2026-11-18
许可 / 类型Licence / kind
arxiv-preprint
许可说明Licence evidence
https://arxiv.org/abs/2610.39302
类目Category
架构Architecture

论文里的数字怎么读How to read the numbers

先分清测什么、怎么算、和谁比、有没有数据污染。拿不准就回原文看实验设置。Before citing a score, separate what it measures, how it is scored, what it compares against and whether contamination is controlled. If unsure, go back to the experiment setup in the paper. 看评测基准See benchmark cards · 看数据集卡See dataset cards

去官网看原页Open the official page →

← 返回AI 论文← Back to AI Papers