---
title: chinese-c4
canonical_url: "https://www.modelscope.cn/datasets/swift/chinese-c4"
md_url: "https://www.modelscope.cn/datasets/swift/chinese-c4.md"
repository: swift/chinese-c4
last_updated: 2024-07-10
license: cc-by-4.0
storage_size: "21 GB"
domain:
  - language
tasks:
  - zh
downloads: 138911
stars: 13
---

# chinese-c4

> chinese-c4 - swift 在 ModelScope 开源的数据集。Chinese-C4 is a clean Chinese internet dataset based on Common Crawl. The dataset is 46.29GB and has undergone multiple cleaning strategies, including Chinese filtering, heuristic cleaning based on punctuation,…

swift/chinese-c4 是 ModelScope 魔搭社区上的zh数据集，涉及 language 领域，存储大小 21 GB，采用 cc-by-4.0 许可。

- **Repository**: swift/chinese-c4
- **License**: cc-by-4.0
- **Tasks**: zh
- **Domain**: language
- **Storage size**: 21 GB
- **Downloads**: 138911
- **Stars**: 13
- **Last updated**: 2024-07-10

Source: https://www.modelscope.cn/datasets/swift/chinese-c4

---

## Introduction

Chinese-C4 is a clean Chinese internet dataset based on Common Crawl. The dataset is 46.29GB and has undergone multiple cleaning strategies, including Chinese filtering, heuristic cleaning based on punctuation, line-based hashing for deduplication, and repetition removal.

The dataset is open source and free for commercial use, and you are welcome to use the data and the cleaning strategies provided and contribute your cleaning strategies.

You can find the cleaning script for the dataset on GitHub [c4-dataset-script](https://github.com/shjwudp/c4-dataset-script).
