JEPA-Anything: Learning Predictive Models across Different Worlds

Taoyong Cui ,

Zhongyao Wang ,

Xinyue Xu ,

Weiyang Liu ,

Zhaochen Yu ,

Yuying Zhang ,

Qiang Gao ,

Mengyue Yang ,

Wanli Ouyang ,

Pheng Ann Heng ,

Yingcheng Wu ,

Zhenfei Yin ,

Ling Yang

AK's Picks Embodied AI and Robotics World Models ML SSLRO AI4S Other AI4S

5093

2026年09月17日

世界建模使智能体能够预判行为后果、指导干预操作，并通过与环境的交互进行学习。然而，当前的预测模型仍局限于特定领域：是否存在一种通用的学习原理，能够支撑跨截然不同系统的统一世界建模？本文提出JEPA-Anything——一种基于正交预测分解（Orthogonal Predictive Factorization, OPF）的、与领域无关的建模框架。该框架在联合嵌入式预测架构（Joint-Embedding Predictive Architecture, JEPA）基础上进一步拓展：OPF将潜在预测目标分解为若干互补因子，通过专用通路分别学习各因子，并在统一的预测架构内将其重新组合。我们在七大领域对JEPA-Anything展开系统性评估，涵盖视觉、生物学、临床病程演化、控制任务、分子动力学、物理场建模以及气象预报。实验内容覆盖表征学习、干预效果预测、分布外泛化能力，以及长时程动力学建模等多个维度，具体包括10项严格配对的动力学任务、对逾1000种临床事件的预测、以及在四种不同分子系统上完成的百步分子轨迹推演（100-step molecular rollouts）。相较于结构匹配的JEPA基线模型，JEPA-Anything在全部10项动力学任务中均显著提升各项报告指标；在干预型弹球（Interventional Pong）任务中，其单次干预预测误差降低34.8%。在全部四种分子系统中，JEPA-Anything均取得最低的一步预测误差与百步预测误差。不仅如此，本框架所识别出的、由特定因子主导的生物学干预方案，已在细胞共培养体系、患者来源类器官、肿瘤组织切片及小鼠模型中获得实验验证；其学习到的隐空间轨道模态亦成功复现出开普勒标度律（Keplerian scaling law），拟合所得斜率达−1.4991。上述结果共同表明：一种共通的、因子化的预测原理，可有效支撑异构世界的统一建模，并将世界建模、主动干预与实证驱动的科学发现紧密联结起来。代码开源地址：https://github.com/Gen-Verse/JEPA-Anything

World modeling enables intelligence to anticipate consequences, guide interventions, and learn from interaction. Yet predictive models remain domain-specific: can a common learning principle support world modeling across radically different systems? We introduce JEPA-Anything, a domain-agnostic framework based on orthogonal predictive factorization (OPF). Extending joint-embedding predictive architectures, OPF decomposes latent targets into complementary factors, learns them through dedicated pathways, and recombines them within a shared predictive design. We evaluate JEPA-Anything across seven domains: vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather. Experiments span representation learning, intervention prediction, out-of-distribution generalization, and long-horizon dynamics, including 10 matched dynamics tasks, forecasting of over 1,000 clinical events, and 100-step molecular rollouts across four systems. Against matched JEPA baselines, JEPA-Anything improves reported metrics on all 10 dynamics tasks and reduces single-intervention prediction error on Interventional Pong by 34.8%. It achieves the lowest one-step and 100-step molecular errors among compared methods in all four systems. Beyond prediction, a factor-nominated biological intervention receives experimental support in cell co-cultures, patient-derived organoids, tumor fragments, and mice; latent orbital modes recover the Keplerian scaling exponent with a fitted slope of -1.4991. These results support a common factorized predictive principle across heterogeneous worlds, connecting world modeling with intervention and experimentally grounded scientific discovery. Code: https://github.com/Gen-Verse/JEPA-Anything

许愿