Proving and Improving the Platonic Representation Hypothesis with JEPAs
Summary
The arXiv paper proves a Platonic Representation Theorem for Joint-Embedding Predictive Architectures (JEPAs) across modalities, showing alignment up to rotation when views are good. It analyzes why language models align with vision only through scale and reveals a scaling law of alignment validated on large multimodal data. The work offers theoretical guarantees and practical guidance for multimodal representation learning.