---
title: "World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning"
canonical_url: "https://www.modelscope.cn/papers/126368"
md_url: "https://www.modelscope.cn/papers/126368.md"
arxiv_id: 2503.10480
published: 2025-03-13
last_updated: 2025-03-13
authors:
  - "Siyin Wang"
  - "Zhaoye Fei"
  - "Qinyuan Cheng"
  - "Shiduo Zhang"
  - "Panpan Cai"
  - "Jinlan Fu"
  - "Xipeng Qiu"
model_name: "Dual Preference Optimization (D2PO)"
model_developer: "复旦大学, 上海创新研究院"
domain:
  - "自然语言处理"
  - "计算机视觉"
  - "深度学习"
  - "机器学习"
type:
  - "自然语言处理"
  - "计算机视觉"
  - "深度学习"
  - "机器学习"
  - "Computation and Language (cs.CL)"
  - "Computer Vision and Pattern Recognition (cs.CV)"
  - "Robotics (cs.RO)"
arxiv_url: "https://arxiv.org/abs/2503.10480"
pdf_url: "https://arxiv.org/pdf/2503.10480.pdf"
---

# World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

> Recent advances in large vision-language models (LVLMs) have shown promise for embodied task planning, yet they struggle with fundamental challenges like dependency constraints and efficiency. Existing approaches either solely optimize action selection or…

「World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning」是 ModelScope 魔搭社区收录的论文，arXiv 2503.10480，作者为 Siyin Wang, Zhaoye Fei, Qinyuan Cheng et al.，发表于 2025-03-13，属于 自然语言处理、计算机视觉、深度学习 领域。

- **ArXiv**: 2503.10480
- **Published**: 2025-03-13
- **Authors**: Siyin Wang, Zhaoye Fei, Qinyuan Cheng, Shiduo Zhang, Panpan Cai, Jinlan Fu, Xipeng Qiu
- **Model**: Dual Preference Optimization (D2PO)
- **Developer**: 复旦大学, 上海创新研究院
- **Domain**: 自然语言处理, 计算机视觉, 深度学习, 机器学习
- **ArXiv URL**: https://arxiv.org/abs/2503.10480
- **PDF**: https://arxiv.org/pdf/2503.10480.pdf

Source: https://www.modelscope.cn/papers/126368

---

> 双剑合璧：通过双偏好优化提升实体任务规划能力

## 摘要

本文针对实体任务规划中的依赖约束和效率问题，提出了一种新的学习框架——双偏好优化（D2PO）。该框架通过偏好学习联合优化状态预测和动作选择，使大型视觉-语言模型（LVLMs）能够更好地理解环境动态，从而提升任务规划能力。与现有方法不同，D2PO不仅优化了动作选择，还通过世界建模增强了模型对环境变化的理解。为了自动收集训练数据，作者引入了一种基于树搜索的机制，通过试错方式探索动作序列并生成逐步偏好数据，避免了人工标注的需求。

在研究背景方面，现有的实体任务规划方法要么仅优化动作选择，要么依赖外部世界模型进行推理，但未能充分利用世界建模的能力来增强规划性能。此外，LVLMs在处理动态交互时存在局限性，主要因为它们缺乏对物理交互动态的建模能力。

在提出的方法中，D2PO通过两种关键组件实现：一是状态预测，即模型根据当前状态和动作预测下一状态；二是动作选择，即基于目标和历史交互选择最优动作。同时，作者设计了一种树搜索算法，用于系统性地探索动作序列，并自动生成用于训练的多模态偏好数据。

实验结果表明，D2PO在VoTa-Bench基准上显著优于现有方法及GPT-4o等基线模型。具体而言，7B参数的模型在成功率和规划效率上分别相对提升了31.4%和33.0%。这些结果验证了D2PO的有效性和潜力，为实体任务规划提供了一种高效且端到端的解决方案。

## Abstract

Recent advances in large vision-language models (LVLMs) have shown promise for embodied task planning, yet they struggle with fundamental challenges like dependency constraints and efficiency. Existing approaches either solely optimize action selection or leverage world models during inference, overlooking the benefits of learning to model the world as a way to enhance planning capabilities. We propose Dual Preference Optimization (D$^2$PO), a new learning framework that jointly optimizes state prediction and action selection through preference learning, enabling LVLMs to understand environment dynamics for better planning. To automatically collect trajectories and stepwise preference data without human annotation, we introduce a tree search mechanism for extensive exploration via trial-and-error. Extensive experiments on VoTa-Bench demonstrate that our D$^2$PO-based method significantly outperforms existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and LLaMA-3.2 (11B), achieving superior task success rates with more efficient execution paths.