---
title: LongWriter-6k
canonical_url: "https://www.modelscope.cn/datasets/ZhipuAI/LongWriter-6k"
md_url: "https://www.modelscope.cn/datasets/ZhipuAI/LongWriter-6k.md"
repository: ZhipuAI/LongWriter-6k
last_updated: 2024-09-12
license: apache-2.0
storage_size: "142 MB"
domain:
  - size_categories
  - task_categories
  - language
tasks:
  - "1K<n<10K"
  - text-generation
  - zh
downloads: 2797
stars: 11
---

# LongWriter-6k

> LongWriter-6k - ZhipuAI 在 ModelScope 开源的数据集。🤗 [LongWriter Dataset] • 💻 [Github Repo] • 📃 [LongWriter Paper]

ZhipuAI/LongWriter-6k 是 ModelScope 魔搭社区上的1K<n<10K、text-generation、zh数据集，涉及 size_categories、task_categories、language 领域，存储大小 142 MB，采用 apache-2.0 许可。

- **Repository**: ZhipuAI/LongWriter-6k
- **License**: apache-2.0
- **Tasks**: 1K<n<10K, text-generation, zh
- **Domain**: size_categories, task_categories, language
- **Storage size**: 142 MB
- **Downloads**: 2797
- **Stars**: 11
- **Last updated**: 2024-09-12

Source: https://www.modelscope.cn/datasets/ZhipuAI/LongWriter-6k

---

# LongWriter-6k

<p align="center">
  🤗 <a href="https://huggingface.co/datasets/THUDM/LongWriter-6k" target="_blank">[LongWriter Dataset] </a> • 💻 <a href="https://github.com/THUDM/LongWriter" target="_blank">[Github Repo]</a> • 📃 <a href="https://arxiv.org/abs/2408.07055" target="_blank">[LongWriter Paper]</a> 
</p>

**LongWriter-6k** dataset contains 6,000 SFT data with ultra-long output ranging from 2k-32k words in length (both English and Chinese). The data can support training LLMs to extend their maximum output window size to 10,000+ words.

## All Models

We open-sourced the following list of models trained on LongWriter-6k:

|Model|Huggingface Repo|Description|
|---|---|---|
|**LongWriter-glm4-9b**| [🤗 Huggingface Repo](https://huggingface.co/THUDM/LongWriter-glm4-9b) | **GLM-4-9B** with an extended 10k+ word output context window |
|**LongWriter-llama3.1-8b**| [🤗 Huggingface Repo](https://huggingface.co/THUDM/LongWriter-llama3.1-8b) | **Llama-3.1-8B** with an extended 10k+ word output context window |

## Citation

If you find our work useful, please consider citing LongWriter:

```
@article{bai2024longwriter,
  title={LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs}, 
  author={Yushi Bai and Jiajie Zhang and Xin Lv and Linzhi Zheng and Siqi Zhu and Lei Hou and Yuxiao Dong and Jie Tang and Juanzi Li},
  journal={arXiv preprint arXiv:2408.07055},
  year={2024}
}
```