---
title: fleurs-r-neucodec-all-languages
canonical_url: "https://www.modelscope.cn/datasets/malaysia-ai/fleurs-r-neucodec-all-languages"
md_url: "https://www.modelscope.cn/datasets/malaysia-ai/fleurs-r-neucodec-all-languages.md"
repository: malaysia-ai/fleurs-r-neucodec-all-languages
last_updated: 2026-10-12
license: cc-by-4.0
storage_size: "118 GB"
downloads: 1069
stars: 0
---

# fleurs-r-neucodec-all-languages

> fleurs-r-neucodec-all-languages - malaysia-ai 在 ModelScope 开源的数据集。FLEURS-R NeuCodec All Languages

malaysia-ai/fleurs-r-neucodec-all-languages 是 ModelScope 魔搭社区上的数据集，存储大小 118 GB，采用 cc-by-4.0 许可。

- **Repository**: malaysia-ai/fleurs-r-neucodec-all-languages
- **License**: cc-by-4.0
- **Storage size**: 118 GB
- **Downloads**: 1069
- **Stars**: 0
- **Last updated**: 2026-10-12

Source: https://www.modelscope.cn/datasets/malaysia-ai/fleurs-r-neucodec-all-languages

---

# FLEURS-R NeuCodec All Languages

FLEURS-R metadata, source audio and precomputed NeuCodec speech tokens for 102
locales, plus a `speaker` label FLEURS itself does not ship.

## Layout

- `data/{locale}-{split}.parquet` — metadata, one row per utterance (this is what the
  viewer shows).
- `audio/{locale}-{split}.zip` — source FLEURS-R audio, 24kHz mono PCM16 WAV, members
  named `audio/{locale}/{split}/{id}.wav` (the `path` column).
- `neucodec/{locale}-{split}-rank{N}.zip` — NeuCodec tokens, members named
  `neucodec/{locale}/{split}/{id}.pt` (the `neucodec_path` column), each a
  `{'source': str, 'codes': IntTensor[1, 1, T]}` torch file at 50 tokens/s.

## Columns

| column | what |
|---|---|
| `id` | FLEURS utterance id |
| `path` | audio member path inside `audio/{locale}-{split}.zip` |
| `neucodec_path` | token member path inside the matching `neucodec/` zip |
| `filename` | bare `{id}.wav` |
| `sentence` | raw transcription |
| `normalized_text` | lowercased, punctuation-stripped transcription |
| `characters` | space-separated character transcription |
| `locale`, `split` | e.g. `en_us`, `train` |
| `gender` | FLEURS speaker gender label |
| `speaker` | **derived** voice identity, `{locale}_spkNN` (see below) |
| `num_samples` | sample count of the original 16kHz FLEURS audio |

## Speaker labels

FLEURS and FLEURS-R ship `gender` but no speaker identity. `speaker` here is derived:
every utterance is embedded with TitaNet-L ([titanet-vectors-fp16](https://github.com/Scicom-AI-Enterprise-Organization/titanet-vectors-fp16)),
and the vectors are clustered per locale with agglomerative clustering
(cosine distance, average linkage, threshold 0.4) **within each gender**, so a
cluster never merges voices FLEURS already labels apart. Clusters smaller than
5 utterances are folded into their nearest centroid, and ids are numbered by
descending cluster size, so `{locale}_spk00` is the most-recorded voice of that locale.

They are clusters, not ground truth: treat them as consistent voice tags for conditioning,
not as verified speaker identities.

Across 102 locales / 279,496 utterances: 5–43 speakers per locale (median 11), mean silhouette 0.60.

## Loading

```python
from datasets import load_dataset

ds = load_dataset("malaysia-ai/fleurs-r-neucodec-all-languages")             # all locales
ds = load_dataset("malaysia-ai/fleurs-r-neucodec-all-languages", "en_us")    # one locale
```

Audio and tokens are the zips above — `path` and `neucodec_path` name the member inside
`audio/{locale}-{split}.zip` and `neucodec/{locale}-{split}-rank*.zip`.

## Source

[google/fleurs-r](https://huggingface.co/datasets/google/fleurs-r) (FLEURS-R, CC BY 4.0),
the restored variant of [google/fleurs](https://huggingface.co/datasets/google/fleurs).