--- title: Bagpiper_SFT_Data canonical_url: "https://www.modelscope.cn/datasets/espnet/Bagpiper_SFT_Data" md_url: "https://www.modelscope.cn/datasets/espnet/Bagpiper_SFT_Data.md" repository: espnet/Bagpiper_SFT_Data last_updated: 2026-10-11 license: other storage_size: "1.0 TB" downloads: 148 stars: 0 --- # Bagpiper_SFT_Data > Bagpiper_SFT_Data - espnet 在 ModelScope 开源的数据集。Release status: the validated Parquet release is being uploaded. The homepage and metadata may appear before every large shard is committed. espnet/Bagpiper_SFT_Data 是 ModelScope 魔搭社区上的数据集，存储大小 1.0 TB，采用 other 许可。 - **Repository**: espnet/Bagpiper_SFT_Data - **License**: other - **Storage size**: 1.0 TB - **Downloads**: 148 - **Stars**: 0 - **Last updated**: 2026-10-11 Source: https://www.modelscope.cn/datasets/espnet/Bagpiper_SFT_Data --- # Bagpiper SFT Data > **Release status:** the validated Parquet release is being uploaded. The > homepage and metadata may appear before every large shard is committed. Bagpiper SFT Data is the supervised fine-tuning corpus for [Bagpiper](https://bagpiper-cmu.github.io/), an open-ended audio language model that understands and generates speech, music, environmental sound, and their mixtures through rich textual captions and planning. The public release has exactly two configurations: | Configuration | Direction | Contents | |---|---|---| | `generation` | natural-language request → planning, rich caption, audio | Open-ended speech, music, sound, and mixed-audio generation sequences | | `understanding` | audio and request → rich caption, reasoning, answer | Audio question answering, multiple-choice reasoning, and transcription sequences | The original source groups are recorded as provenance inside each row; they do not create additional public configurations. ## Prepare for ESPnet training The complete [ESPnet-format release](https://huggingface.co/datasets/espnet/Bagpiper_SFT_Data_ESPnet) preserves all 2,667,017 source rows, ordered text turns, and original encoded audio bytes. It converts this Parquet schema into three-element dialogue messages and portable audio files. Source `row_id` becomes the globally unique ESPnet `example_id`; repeated training examples remain present. ```bash python -m pip install huggingface_hub zstandard hf download espnet/Bagpiper_SFT_Data_ESPnet prepare_espnet.py --local-dir tools python tools/prepare_espnet.py \ --repo-id espnet/Bagpiper_SFT_Data_ESPnet --data-root /path/to/bagpiper-general-sft ``` The helper verifies archive/audio hashes, resolves local paths, and writes `prepared//{all,train,valid}.json` plus `dialogues.jsonl`. Add `--max-shards 2` for a small trial and repeat `--partition NAME` to select applications. Audio SHA256 determines the suggested 1% validation split; `all.json` retains the original training pool. The split is not the paper's official split. Prepare token-length statistics with the checkpoint-compatible configuration as shown in the converted release's README. See its [full release validation](https://huggingface.co/datasets/espnet/Bagpiper_SFT_Data_ESPnet/blob/main/RELEASE_VALIDATION.json) and [public download/ESPnet reader checks](https://huggingface.co/datasets/espnet/Bagpiper_SFT_Data_ESPnet/blob/main/PUBLIC_DOWNLOAD_VALIDATION.json). This conversion preserves the original source selection and applicable terms. ## Loading the data Named Hugging Face configurations are deferred. Load a partition with an explicit Parquet glob; streaming is recommended for this 1.13 TB release: ```python from datasets import load_dataset generation = load_dataset( "parquet", data_files={"train": "hf://datasets/espnet/Bagpiper_SFT_Data/generation/*.parquet"}, split="train", streaming=True, ) understanding = load_dataset( "parquet", data_files={"train": "hf://datasets/espnet/Bagpiper_SFT_Data/understanding/*.parquet"}, split="train", streaming=True, ) example = next(iter(generation)) audio_bytes = example["audio"]["bytes"] ``` Every row will be self-contained: the ordered text conversation and complete encoded audio bytes are stored in the same Parquet row. No external media download, filesystem path, or table join will be required. Data will be split across many deterministic Parquet shards targeting approximately 256 MiB each. See [Schema](docs/SCHEMA.md) for the planned row format and [Release status](docs/RELEASE_STATUS.md) for the publication gates. ## Data construction Bagpiper SFT examples are synthesized from rich-captioned audio. Understanding sequences combine an input audio clip and task request with a rich caption, reasoning trace, and answer. Generation sequences reverse the direction: a request is expanded into planning and a rich caption before the target audio. The project uses large language models for request/reasoning simulation and quality filtering. ## Release statistics | Partition | Rows | Shards | Parquet bytes | Embedded audio bytes | |---|---:|---:|---:|---:| | `generation` | 1,474,011 | 1,944 | 363,645,830,327 | 411,563,826,060 | | `understanding` | 1,193,006 | 4,860 | 768,716,849,416 | 1,251,578,476,222 | | **Total** | **2,667,017** | **6,804** | **1,132,362,679,743** | **1,663,142,302,282** | | Partition | `source_subset` | Rows | |---|---|---:| | generation | `part2_gen_v1_realistic` | 463,850 | | generation | `part2_gen_v1_imaginary` | 371,827 | | generation | `part3_gen_v1_realistic` | 95,134 | | generation | `part3_gen_v1_imaginary` | 72,508 | | generation | `part4_gen_v1_realistic` | 268,584 | | generation | `part4_gen_v1_imaginary` | 202,108 | | understanding | `airbench_train_v1` | 357,896 | | understanding | `mmau_train_v1` | 334,224 | | understanding | `asr_v2_inverse_200k` | 200,000 | | understanding | `audiobench_train_v1` | 300,886 | Seven understanding rows sharing one header-only, zero-frame `FoR_4308` WAV were removed after content audit. Intended reuse of valid audio across multiple rows is retained. ## Known limitations - Captions, reasoning, and quality judgments are machine-generated and can contain hallucinations or biases. - The Bagpiper paper reports audio-fingerprint overlap between its SFT inputs and portions of AudioBench (12.8%) and AIR-Bench (3.2%). Results on those benchmarks should be interpreted with this disclosure. - Source audio has heterogeneous provenance and licensing. Configurations or examples without confirmed redistribution rights will be withheld rather than assigned an unsupported blanket license. - The corpus may include synthetic or transformed audio and should not be treated as verified human annotation. - The current understanding partition follows the checked-in recipe's v1 mixture. It is larger than the approximately 845k understanding sequences reported by the paper; paper-v2 alignment is deferred by owner decision. - The corpus is not exhaustively moderated for personal information, unsafe content, offensive language, or copyrighted text embedded in transcripts. ## Related resources - Paper: [Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions](https://openreview.net/forum?id=FuHs64E3X6) - Project: [bagpiper-cmu.github.io](https://bagpiper-cmu.github.io/) - Base model: [espnet/bagpiper](https://huggingface.co/espnet/bagpiper) - Pretraining data: [espnet/Bagpiper_PreTrain_Data](https://huggingface.co/datasets/espnet/Bagpiper_PreTrain_Data) - ESPnet: [espnet/espnet](https://github.com/espnet/espnet) ## Citation Please cite the accepted Bagpiper paper when using this dataset: ```bibtex @inproceedings{anonymous2026bagpiper, title={Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions}, author={Jinchuan Tian and Haoran Wang and Bo-Hao Su and Chien-yu Huang and Qingzheng Wang and Jiatong Shi and William Chen and Xun Gong and Siddhant Arora and Chin-Jou Li and Masao Someki and Takashi Maekaku and Keita Goto and Yusuke Shinohara and Jin Sakuma and Chao-Han Huck Yang and Shinji Watanabe}, booktitle={Third Conference on Language Modeling}, year={2026}, url={https://openreview.net/forum?id=FuHs64E3X6} } ``` ## Contact and takedown Please use the repository community tab for data issues, provenance corrections, or takedown requests. Include the configuration and `example_id`; do not repost sensitive media in the report.