---
title: "WritingBench: A Comprehensive Benchmark for Generative Writing"
canonical_url: "https://www.modelscope.cn/papers/124147"
md_url: "https://www.modelscope.cn/papers/124147.md"
arxiv_id: 2503.05244
published: 2025-03-07
last_updated: 2025-03-07
authors:
  - "Yuning Wu"
  - "Jiahao Mei"
  - "Ming Yan"
  - "Chenliang Li"
  - "SHaopeng Lai"
  - "Yuran Ren"
  - "Zijia Wang"
  - "Ji Zhang"
  - "Mengyue Wu"
  - "Qin Jin"
  - "Fei Huang"
model_name: WritingBench
model_developer: "阿里巴巴集团, 中国人民大学, 上海交通大学"
domain:
  - "自然语言处理"
  - "深度学习"
  - "机器学习"
type:
  - "自然语言处理"
  - "深度学习"
  - "机器学习"
  - "Artificial Intelligence (cs.AI)"
  - "Computation and Language (cs.CL)"
arxiv_url: "https://arxiv.org/abs/2503.05244"
pdf_url: "https://arxiv.org/pdf/2503.05244.pdf"
---

# WritingBench: A Comprehensive Benchmark for Generative Writing

> Recent advancements in large language models (LLMs) have significantly enhanced text generation capabilities, yet evaluating their performance in generative writing remains a challenge. Existing benchmarks primarily focus on generic text generation or…

「WritingBench: A Comprehensive Benchmark for Generative Writing」是 ModelScope 魔搭社区收录的论文，arXiv 2503.05244，作者为 Yuning Wu, Jiahao Mei, Ming Yan et al.，发表于 2025-03-07，属于 自然语言处理、深度学习、机器学习 领域。

- **ArXiv**: 2503.05244
- **Published**: 2025-03-07
- **Authors**: Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, SHaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, Fei Huang
- **Model**: WritingBench
- **Developer**: 阿里巴巴集团, 中国人民大学, 上海交通大学
- **Domain**: 自然语言处理, 深度学习, 机器学习
- **ArXiv URL**: https://arxiv.org/abs/2503.05244
- **PDF**: https://arxiv.org/pdf/2503.05244.pdf

Source: https://www.modelscope.cn/papers/124147

---

> WritingBench：构建全面的生成性写作评估基准

## 摘要

近年来，大型语言模型（LLMs）显著提升了文本生成能力，但在评估其在生成性写作中的表现时仍面临挑战。现有的基准测试主要集中在通用文本生成或有限的写作任务上，未能捕捉到高质量书面内容在不同领域的多样化需求。为了解决这一问题，本文提出了WritingBench，一个全面的基准测试框架，旨在评估LLMs在6个核心写作领域和100个子领域的表现，涵盖了创意、说服力、信息性和技术性写作。该框架进一步提出了一种基于查询的评估方法，使LLMs能够动态生成实例特定的评估标准，并通过微调的批评模型进行评分，从而在风格、格式和长度方面进行评估。实验结果表明，该框架的数据整理能力使得7B参数的模型接近最先进水平。WritingBench开源了基准测试工具和模块化框架组件，以推动LLMs在写作领域的进一步发展。

## Abstract

Recent advancements in large language models (LLMs) have significantly enhanced text generation capabilities, yet evaluating their performance in generative writing remains a challenge. Existing benchmarks primarily focus on generic text generation or limited in writing tasks, failing to capture the diverse requirements of high-quality written contents across various domains. To bridge this gap, we present WritingBench, a comprehensive benchmark designed to evaluate LLMs across 6 core writing domains and 100 subdomains, encompassing creative, persuasive, informative, and technical writing. We further propose a query-dependent evaluation framework that empowers LLMs to dynamically generate instance-specific assessment criteria. This framework is complemented by a fine-tuned critic model for criteria-aware scoring, enabling evaluations in style, format and length. The framework's validity is further demonstrated by its data curation capability, which enables 7B-parameter models to approach state-of-the-art (SOTA) performance. We open-source the benchmark, along with evaluation tools and modular framework components, to advance the development of LLMs in writing.