---
title: WebInstructSub
canonical_url: "https://www.modelscope.cn/datasets/swift/WebInstructSub"
md_url: "https://www.modelscope.cn/datasets/swift/WebInstructSub.md"
repository: swift/WebInstructSub
last_updated: 2024-10-27
license: apache-2.0
storage_size: "3.3 GB"
domain:
  - size_categories
  - dataset_info
  - language
  - task_categories
  - pretty_name
tasks:
  - "1M<n<10M"
  - dataset_size
  - en
  - question-answering
  - WebInstruct
downloads: 3494
stars: 2
---

# WebInstructSub

> WebInstructSub - swift 在 ModelScope 开源的数据集。🦣 MAmmoTH2: Scaling Instructions from the Web

swift/WebInstructSub 是 ModelScope 魔搭社区上的1M<n<10M、dataset_size、en数据集，涉及 size_categories、dataset_info、language 领域，存储大小 3.3 GB，采用 apache-2.0 许可。

- **Repository**: swift/WebInstructSub
- **License**: apache-2.0
- **Tasks**: 1M<n<10M, dataset_size, en, question-answering, WebInstruct
- **Domain**: size_categories, dataset_info, language, task_categories, pretty_name
- **Storage size**: 3.3 GB
- **Downloads**: 3494
- **Stars**: 2
- **Last updated**: 2024-10-27

Source: https://www.modelscope.cn/datasets/swift/WebInstructSub

---

# 🦣 MAmmoTH2: Scaling Instructions from the Web

Project Page: [https://tiger-ai-lab.github.io/MAmmoTH2/](https://tiger-ai-lab.github.io/MAmmoTH2/)

Paper: [https://arxiv.org/pdf/2405.03548](https://arxiv.org/pdf/2405.03548)

Code: [https://github.com/TIGER-AI-Lab/MAmmoTH2](https://github.com/TIGER-AI-Lab/MAmmoTH2)

## WebInstruct (Subset)

This repo contains the partial dataset used in "MAmmoTH2: Scaling Instructions from the Web". This partial data is coming mostly from the forums like stackexchange. This subset contains very high-quality data to boost LLM performance through instruction tuning.

## License

- For the data from "mathstackexchange" and "stackexchange", we use Apache-2.0 license. You are free to share and adapt for any purposes.
- For the data from "socratic", we use CC BY-NC 4.0 license according to https://socratic.org/terms. You are free to share and adapt, but only for non-commercial purposes.

## Fields in our dataset

The field `orig_question' and `orig_answer' are the extracted question-answer pairs from the recalled documents. The `question' and `answer' are the refined version of the extracted question/answer pairs.

Regarding the data source:
1. mathstackexchange: https://math.stackexchange.com/.
2. stackexchange: including https://physics.stackexchange.com/, https://biology.stackexchange.com/, https://chemistry.stackexchange.com/, https://cs.stackexchange.com/.
3. Socratic: the data is originally from https://socratic.org/.

## Size of different sources

| Domain               |  Size    |   Subjects                                                                                |
|:---------------------|:---------|:------------------------------------------------------------------------------------------|
| MathStackExchange    | 1484630  | Mathematics                                                                               |
| ScienceStackExchange | 317209   | Physics, Biology, Chemistry, Computer Science                                             |
| Socratic	           | 533384	  | Mathematics, Science, Humanties                                                           |

## Dataset Construction

We propose discovering instruction data from the web. We argue that vast amounts of high-quality instruction data exist in the web corpus, spanning various domains like math and science. Our three-step pipeline involves recalling documents from Common Crawl, extracting Q-A pairs, and refining them for quality. This approach yields 10 million instruction-response pairs, offering a scalable alternative to existing datasets. We name our curated dataset as WebInstruct.

![Project Framework](https://tiger-ai-lab.github.io/MAmmoTH2/static/images/teaser.jpg)

## Citation
```
@article{yue2024mammoth2,
  title={MAmmoTH2: Scaling Instructions from the Web},
  author={Yue, Xiang and Zheng, Tuney and Zhang, Ge and Chen, Wenhu},
  journal={Advances in Neural Information Processing Systems},
  year={2024}
}
```
