NarrativeFlow: Flow-Based Vision-Language-Action Model Using Robot Velocity Fields

Embodied AI and Robotics MLMRD IL ML Generative Models
我们聚焦于语言驱动的基于流场的机器人操控任务,其中“机器人流”(即机器人速度场)作为一种与具体机器人本体无关、以运动为核心的表征方式,可有效利用从多种机器人平台采集的数据。该任务至关重要,因为语言驱动的操控能力是实用化机器人系统不可或缺的核心功能;然而,当前机器人基础模型的规模化发展仍受限于针对特定机器人本体、费时费力的数据采集过程。现有方法要么仅用稀疏关键点位移对机器人流进行粗略近似,要么无法支持语言驱动的操控任务。为克服这一局限,我们提出了NarrativeFlow:该方法采用以语言为条件的流匹配(flow-matching)建模范式,将机器人流表示为连续的速度场,从而生成在物理上与真实世界操控过程高度一致的机器人运动轨迹。为验证NarrativeFlow的有效性,我们在多个标准的语言驱动操控数据集上开展了实验评估。结果表明,NarrativeFlow在各项标准评测指标上均显著优于代表性基线方法。此外,通过真实机器人平台上的实际操控实验,我们进一步证实:NarrativeFlow在多项操控任务中均取得了高于基线方法的成功率。项目主页详见:https://shota0520.github.io/NarrativeFlow-project-page/
We focus on language-conditioned flow-based manipulation, where robot flows (robot velocity fields) serve as embodiment-agnostic, motion-centric representations for leveraging data collected from multiple robot platforms. This task is crucial because language-conditioned manipulation is essential for practical robotic systems, yet scaling robot foundation models remains limited by the labor-intensive collection of embodiment-specific data. Existing methods either coarsely approximate robot flows with sparse keypoint displacements, or cannot handle language-conditioned manipulation. To address this limitation, we propose NarrativeFlow, which models robot flows as continuous velocity fields using a flow-matching formulation conditioned on language. Accordingly, NarrativeFlow generates robot flows that are physically consistent with real-world manipulation. To validate NarrativeFlow, we have conducted experiments on standard datasets for language-conditioned manipulation. The experimental results show that NarrativeFlow outperforms representative baseline methods on standard evaluation metrics. Furthermore, through real-world experiments, we show that NarrativeFlow achieves higher success rates than baseline methods across multiple manipulation tasks. The project page is available at https://shota0520.github.io/NarrativeFlow-project-page/
许愿