---
title: laion-voice-profiles-sft
canonical_url: "https://www.modelscope.cn/datasets/laion/laion-voice-profiles-sft"
md_url: "https://www.modelscope.cn/datasets/laion/laion-voice-profiles-sft.md"
repository: laion/laion-voice-profiles-sft
last_updated: 2026-10-26
license: cc-by-4.0
storage_size: "5.8 GB"
downloads: 261
stars: 0
---

# laion-voice-profiles-sft

> laion-voice-profiles-sft - laion 在 ModelScope 开源的数据集。LAION Voice Profiles — TTS supervised fine-tuning set

laion/laion-voice-profiles-sft 是 ModelScope 魔搭社区上的数据集，存储大小 5.8 GB，采用 cc-by-4.0 许可。

- **Repository**: laion/laion-voice-profiles-sft
- **License**: cc-by-4.0
- **Storage size**: 5.8 GB
- **Downloads**: 261
- **Stars**: 0
- **Last updated**: 2026-10-26

Source: https://www.modelscope.cn/datasets/laion/laion-voice-profiles-sft

---

# LAION Voice Profiles — TTS supervised fine-tuning set

**Authors: Christoph Schuhmann and LAION.**

**1,200,531 instruction-tuning samples for reference-conditioned TTS**, drawn from 500 synthetic
voice profiles: for each of the 842 acting conditions of each voice, the **best 3 of its 48
candidate takes**, each paired with a **different clip of the same voice from another group** as
the reference, plus the exact conditioning prompt, MOSS audio codes for both target and reference,
word-level forced alignment, and the full corrected annotation.

Every record carries **two prompts** — one conditioned on reference audio + caption, one on the
speaker's **name** — so the same data trains voice cloning and voice memorisation.

* **1,200,531 samples** · **500 voices** · 2,401 per voice · 6.3 GB · en + de
* target and reference shipped as **MOSS-Audio-Tokenizer-v2 codes**, not waveforms
* DPO companion: [`laion/laion-voice-profiles-dpo`](https://huggingface.co/datasets/laion/laion-voice-profiles-dpo)

---

## Read this first

1. **This is codes, not audio.** Targets and references are MOSS RVQ codes (`(frames, 12)`
   `uint16`). That is what the base model consumes and emits. Decode with the codec if you want to
   listen. `uid` joins to the waveform in `laion/laion-voice-profiles-annotated`.
2. **The reference is deliberately never the profile's own reference clip**, and never a clip from
   the same group. It is another take of the same voice, usually a different emotion, chosen fresh
   per record so the model cannot learn a fixed pairing.
3. **WER was not used to select anything**, on purpose — see §"Ranking".
4. **Speaker identity against the *nominal* reference is weak corpus-wide, and that is a property
   of the source material, not of this selection.** Per-voice mean `spk_sim` has median **0.445**
   (range 0.176 to 0.691), and the median voice has only ~**20 %** of its takes at `spk_sim >= 0.60`.
   The takes of one voice are much more like *each other* than like the profile's reference clip —
   which is why a within-voice reference at cosine 0.60 is easy to find while 0.60 against the
   nominal reference is not. Train on this and you get the voice the generator actually produced,
   which is consistently *a* voice but often not the reference clip's. `spk_sim` and `ref_spk_sim`
   are on every row.
5. Nobody has listened to this in a controlled study. Every quality number is a model output.

---

## Files, layout and where the audio is

| path | count | size | what it is |
|---|--:|--:|---|
| `data/part-00000.parquet` … `part-00499.parquet` | 500 | 6.26 GB | the samples — one file per voice profile, 72 columns |
| `code/` | 13 | — | the build: `build_voice.py`, `emit.py`, `run_seq.py`, `run_all.sbatch`, `aggregate.py`, `make_cards.py`, `namepools.py`, `capgate.py`, `quickstart.py`, … |
| `names.csv`, `names.json` | 2 | — | voice id → speaker name, gender class, and the `card_gender` disagreement flags |
| `per_voice_stats.csv` | 1 | — | per-voice realised counts and score distributions |
| `verify.json`, `link_check.json`, `VERIFY.md` | 3 | — | the verification run reproduced below |

**There is no audio in this repository — no waveform, no MP3, no `audio` column, no tar.** Target
and reference are **MOSS-Audio-Tokenizer-v2 codes**: `target_codes` / `target_frames` and
`ref_codes` / `ref_frames`, raw little-endian `uint16`, reshaped to `(frames, 12)`. `emo_vector`
and `vn_reg_vector` are the other two `binary` columns and are packed `float32` score buffers, not
audio.

For the waveform, go to
[`laion/laion-voice-profiles-annotated`](https://huggingface.co/datasets/laion/laion-voice-profiles-annotated):
this repository's `uid` **is** the member stem of that repository's tars, and `shard`
(`vprof_base-NNNNN`) names the tar under `data/origin=original/`.

```python
import glob, io, tarfile, numpy as np, pyarrow.parquet as pq
row = pq.read_table("data/part-00000.parquet").slice(0, 1).to_pylist()[0]
codes = np.frombuffer(row["target_codes"], dtype="<u2").reshape(row["target_frames"], row["n_vq"])
# the same take's 48 kHz MP3, in the annotation corpus:
#   data/origin=original/{row['shard']}.tar  ->  member  {row['uid']}.mp3
```

**Full column reference:** every one of the 72 columns, with its type and meaning, plus the
dimension each of the 57 VoiceNet codes stands for, is in **[`COLUMNS.md`](COLUMNS.md)**.

---

## Ranking — how the 3 of 48 were chosen

```
score = norm(genuineness) + norm(blend) + 2 * norm(emo_strength)
```

**Emotion strength counts double**, by design: the point of this corpus is acting, and
genuineness and burst blend are means to it.

**`norm` is a per-voice quantile (rank) transform** — `rank(x) / (n - 1)` over all valid takes of
that voice, giving each term a uniform [0, 1] marginal before summing. Per-voice, because the
three scores have visibly different scales and spreads from voice to voice (a per-voice mean
`spk_sim` ranges from 0.20 to 0.86 across the 500, and `blend` is heavily right-skewed), so a
global z-score would be dominated by the loudest voices and by `blend`'s tail. The rank transform
makes the "2x" weight mean exactly what it says: emotion strength gets twice the *ordinal* pull.
The unnormalised inputs (`genuineness`, `blend`, `emo_strength`) and the normalised terms
(`n_genuineness`, `n_blend`, `n_emo_strength`) are both shipped, so you can re-rank differently
without re-reading the corpus.

**WER is ignored.** The corpus was re-annotated end to end, and the generation-time `wer` compares
a whisper transcript against a prompt that deliberately contains vocal-burst tags and non-verbal
direction; a take that hits the direction perfectly can score a terrible WER. It is shipped
(`wer`) but was not consulted.

Takes are eligible if they are non-empty, at least 0.5 s, and present in the tokenised corpus.

---

## Reference-clip selection

For each sample, the reference is **another take of the same voice, from a different `gid`**,
required to satisfy

```
cos(ECAPA(sample), ECAPA(reference)) >= 0.60
```

`spk_emb` is the generator's own 192-d ECAPA embedding. It was verified against the corpus's
`spk_sim` column before use: recovering the profile's reference embedding by least squares from
40k takes reproduces `spk_sim` with **correlation 1.00000 and RMSE 1e-5**, so this is exactly the
space the project's speaker thresholds live in.

**Why 0.60.** It is the project's own `MIN_SPK_TRAIN`, and it is where the null distribution dies:
measured over **44.2 million** genuinely cross-voice pairs (24 voices, all pairs),
`P(cos >= 0.60) = 0.0042`. So a reference admitted at this threshold has a **0.42 % chance** of
being what the corpus would call a different speaker. At 0.40 — the corpus's "not the same
speaker" floor — that rises to 3.6 %; at 0.70 it falls to 0.09 % but coverage drops sharply.

**There is deliberately no floor on the reference's own `spk_sim`.** The requirement is similarity
to the *current sample*, not to the profile's nominal reference. An early build gated the pool at
`spk_sim >= 0.60` as well and collapsed the low-identity voices — `emolia_c0241` produced **6**
samples instead of ~2,400, because 0.005 % of its takes reach 0.60 against its own reference —
while changing the healthy voices by nothing at all. `ref_spk_sim` is shipped so you can re-impose
that gate if you want it.

**Varying the pairing.** The pool is up to 3 takes per `gid` (highest `spk_sim` first), 2–25 s,
capped at 1,500 per voice. Assignment walks the samples in random order and gives each the
*least-used* eligible reference, ties broken at random, so the load spreads: about
**1,297 distinct reference clips per voice** serve 2,401 samples, and the
mean realised cosine is **0.659**. `ref_emotion` / `ref_block` differ from the target's on
the large majority of rows — different emotion is the normal case, as intended.

**Coverage.** For each `gid` the top **8** candidates by score are considered and the best **3**
that can be paired with a valid reference are kept, so a candidate that is too far off-identity
for any reference to match is replaced by the next-best take rather than dropping the slot.
Realised: 95.1 % on average (median 95.3 %, worst voice 86.8 %) of the nominal 842 x 3 = 2,526 per voice. The shortfall is concentrated in
voices whose takes are mutually inconsistent; it is not uniform.

---
## The material this is built from

500 synthetic **voice profiles**. Each profile is one reference speaker driven through the *same*
fixed matrix of acting conditions in English and German, keeping **every** candidate take rather
than only the winners. Generator
[`laion/moss-tts-local-transformer-4.55b-voice-acting-v2`](https://huggingface.co/laion/moss-tts-local-transformer-4.55b-voice-acting-v2),
codec [`OpenMOSS-Team/MOSS-Audio-Tokenizer-v2`](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-v2),
run tag `PPILOT2`. The full 20,125,736-take corpus with audio is
[`laion/laion-voice-profiles-annotated`](https://huggingface.co/datasets/laion/laion-voice-profiles-annotated);
the per-voice LoRAs and reference clips are
[`laion/moss-voice-profile-loras-500`](https://huggingface.co/laion/moss-voice-profile-loras-500).

### The group structure, measured

Per voice: **842 condition groups (`gid`)**, of which **832 have 48 takes** and 10 have 32.
Each `gid` splits into **subset A = 32 takes** and **subset B = 16 takes**.

Two things about this are easy to get wrong, and both were checked against the data rather than
taken from prose:

* **A `gid` is not one sentence.** All 48 takes of a `gid` carry *different* text — 807 of 842
  `gid`s have 48 distinct sentences. `text_slot` is a slot label (`char::granite-titan`), not a
  sentence. The one-sentence-many-takes reading of a `gid` is wrong.
* **The minimal pairs live in subset B, across `gid`s.** The same sentence is reused at
  `(voice, parent_key, lang, sub_idx)` — i.e. the same B-slot of every `gid` under one
  `parent_key`. That yields ~**3,055 same-sentence quadruples per voice**: the same line rendered
  at the 4 levels of one VoiceNet dimension, or under the 4 acting conditions (A/B/C/D) of one
  emotion. Nothing else in the corpus is a same-sentence set.

The "832" figure that circulated for this corpus is exactly right — it is the 832 full-size
`gid`s — and "16" and "32" are subsets B and A of each of them, not group sizes. 832 x 3 x 500
= 1,248,000 is the arithmetic the ~1.25 M target came from.

### `condition` and `level`

Emotion cells are `E|<Emotion>|<A|B|C|D>`, where A/B/C/D is the 2x2 of **intensity x containment**:

| code | intensity | containment | prompt gloss |
|---|---|---|---|
| `A` | intense | free | maximum emotion, freely let out |
| `B` | moderate | free | mild emotion, freely expressed |
| `C` | intense | contained | very strong but held back — it leaks through |
| `D` | moderate | contained | mild, quietly controlled |

VoiceNet cells are `V|<DIM>|<level>` over the ladder
`extremely_low < moderately_low < moderately_high < very_high`.

---

## Provenance of every number, and which annotation tree was used

This corpus has a known defect: the annotation layer of the **live** release
`laion-tts-annotated-v1/index/vprof_base` was computed on a **half-speed decode** — a stereo mp3
bug doubled the sample count. Every duration, every emotion intensity and all 57 VoiceNet
dimensions in that tree are wrong. Nothing in this dataset comes from it. Two independent checks
confirm which tree is which, both run for this build:

* `dur_s` in the live tree is **exactly 2.000x** the re-annotated `dur_s` on all 9,916 rows of
  shard 0 (std 0.000).
* the re-annotated `dur_s` equals the **generator's own** `dur` to the digit (ratio 1.000, std 0).
  The generator measured it on the in-memory waveform *before* mp3 encode, so it never saw the bug.

**The MOSS codes were re-tokenised too, and that was checked on the bytes, not on the index.**
For the same six utterances, the `.moss.npy` member of the re-annotated tar has shape
`(199, 12)`, `(170, 12)`, `(186, 12)`, `(400, 12)`, `(186, 12)`, `(159, 12)` — exactly
`round(generator_dur * 12.5)` in every case — while the live tree's tar has
`(398, 12)`, `(340, 12)`, `(372, 12)`, `(800, 12)`, `(372, 12)`, `(318, 12)`, i.e. double.
Across shard 0, `moss_frames / (generator_dur * 12.5)` is **1.000 (std 0)** in the re-annotated
tree and **2.000 (std 0)** in the live one. The codes shipped here are the corrected ones. This
matters more than any other check in this build: the earlier half-speed MOSS codes are unusable,
and an index-only comparison could not have caught it — `moss_frames == round(dur_s * 12.5)`
passed at 100 % on the broken data precisely because both terms were doubled.

Accordingly:

| field group | source | why |
|---|---|---|
| `genuineness`, `blend`, `emo_strength`, `spk_sim`, `spk_emb`, `dur`, `wer` | the generator's own `meta-*.parquet` (shipped verbatim as `index/vprof_base_scores`) | measured pre-encode; unaffected by the decode bug |
| `emotion`, `condition`, `dim`, `level`, `cond_key`, `parent_key`, `subset`, `sub_idx` | same | these exist **nowhere else** |
| `instruction_generation` | the generator's `caption` column | the literal conditioning string the model was given |
| 40 `emo_*`, 57 `vn_*_reg`, 4 `qual_*`, `blend_0_10`, `genuineness_0_6`, `n_bursts`, `text`, `caption_general` | the corrected re-annotation tree `laion-tts-annotated-v1-reann` | recomputed on correctly decoded audio |
| `words_json` | `<uid>.json` in the re-annotated tars — MMS_FA forced alignment of the prompt text | |
| `*_codes` | `<uid>.moss.npy` in the re-annotated tars | |

---

## The procedural caption is regenerated here, not copied

`instruction_caption` is **rendered in this build from the numeric columns**. The corpus's own
caption column is deliberately **not shipped**, because it was being regenerated corpus-wide while
this dataset was built and copying it would have baked a caption into a training set that
disagrees with the corpus a downstream user reads. Two independent defects are fixed:

**1. Polarity.** The GEND and BKGN prose ladders ran backwards — high `vn_GEND_reg` is
**masculine** and high `vn_BKGN_reg` is **cleaner**, and the ladders said the opposite. In the
live corpus tree the mean `vn_GEND_reg` of clips captioned *masculine* is **0.81** against **2.49**
for *feminine*. Captions here use the corrected ladders (`caption_render.py`, `polarity="v2"` —
the same tables as `caption2.py`, md5 `ec55b223bab60f9f1cfb7b9bedfdf76f`). The numeric columns
were always correct; only the words inverted. The polarity is now confirmed four independent ways:
chest-resonance correlation (+0.87), head-resonance (−0.55), the corrected ladder's own
derivation, and **89.2 % agreement between numeric `vn_GEND_reg` and each profile's independent
design-spec `card_gender`** — a figure that would be ~11 % if the polarity were inverted.

**2. The emotion gate.** Which emotions a caption *names* is decided here by a **percentile**
gate rather than an absolute threshold: an emotion is named only if its value lands in the top
**10 %** of that emotion's own distribution, at most the top 3, and nothing at all if none clears
the floor. The distribution comes from the tie-aware mid-rank ECDF in
`$NB/emonorm/out/capnorm.npz` — 4,096-bin histograms over **132,833,726** rows pooled across 8
datasets and both languages. This matters because the 40 emotion heads sit on very different
scales: measured on 120,659 rows of this corpus, `emo_Interest` has median **1.969** and no
values at or below zero, while `emo_Infatuation` has median **−0.015** and **81.7 %** of its
values at or below zero. One absolute threshold cannot be right for both.

Tie-awareness is what makes this work rather than a plain 0.90-quantile value threshold: several
heads have a majority point mass at zero, so a value threshold would sit at zero and admit
everything, while mid-rank gives the whole tied group a percentile near 0.5 and correctly declines
to name it. Implementation and validation runs are in `code/capgate.py`.

**One claim about this that I could not reproduce, stated because it shaped the design.** The
percentile gate was introduced on the premise that the absolute gate named *Interest* on ~95 % of
captions. On this corpus that is not what happens: the shipped `caption_general` of the
re-annotated tree names "interest" on **2.3 %** of 120,659 sampled rows. So the gate here is a
principled normalisation across incommensurable heads — which the median/zero figures above do
justify — and **not** a fix for a runaway-Interest bug, which this corpus does not exhibit. If you
prefer the corpus's own wording, regenerate it with `caption_render.render(row, polarity="v2")`;
every column it needs is shipped.

**Realised on the captions actually shipped here**, measured on 95,680 of them: **Interest
4.4 %** of clips, all **40/40** emotion heads appear, **2.93 emotions
named per clip**, and **0.00 %** carry "no dominant emotion". That last figure is essentially
zero because of selection, not because of the gate: every row here is one of the top-3 takes of
its cell ranked with
**emotion strength weighted double**, so essentially every clip clears a top-10 % floor on
something. Do not read these numbers as corpus statistics — they are statistics of a deliberately
emotional subset. `instruction_caption` is nullable and is null on 2 of the sampled rows
where the renderer could not build a caption from the row's numbers.

**The prompts were never affected by any of this.** `prompt_reference` and `prompt_name` embed
`instruction_generation` — the generator's own `GENERAL:`/`SCRIPT:` conditioning string, which is
what the model was actually given at generation time and is not a procedural caption at all. Swap
`instruction_caption` into the `- Instruction:` slot to train caption-following instead.

---

## MOSS codes — layout, and the "32 tokens" question

`target_codes` / `chosen_codes` / `rejected_codes` / `ref_codes` / `donor_codes` are raw
**little-endian `uint16`**, shape `(frames, 12)` after reshape.

```python
import numpy as np
codes = np.frombuffer(row["target_codes"], dtype="<u2").reshape(row["target_frames"], row["n_vq"])
```

* **12 codebooks x 1024 entries, 12.5 fps.** `frames == round(dur_s * 12.5)` holds on every row.
* **The atomic unit is one frame = 12 tokens = 80 ms.** There is no 32-token block, no delay
  pattern and no interleaving: the model emits all 12 channels of one frame sequentially inside a
  1-layer local ("depth") transformer with `n_positions = n_vq + 1 = 13`, RQ-Transformer style.
* The **32** associated with this stack is the *codec's* `num_quantizers: 32`. The codec ships 32
  residual quantizers; the TTS model consumes only the **first 12** (`codes[:, :n_vq]`). It is a
  depth, not a block size. Every cut in this dataset is therefore aligned to a **12-token frame
  boundary**, which is the only alignment the format actually has.

---

## The two conditioning modes

Every record carries **both**, as complete strings, in the base model's own `<user_inst>` template
(`processing_moss_tts.py::UserMessage`), and also as the components so you can re-template:

**`prompt_reference`** — reference audio + caption. `<|audio|>` is where the reference clip's MOSS
codes are spliced in as a prefix (`ref_codes`, `ref_frames`):

```
<user_inst>
- Reference(s):
<|audio|>
- Instruction:
GENERAL: A voice intensely expressing affection, ... impossible to hide.
SCRIPT:
(letting it out / not hiding it, warm and open, unguarded) "..."
- Tokens:
19
- Quality:
None
- Sound Event:
None
- Ambient Sound:
None
- Language:
German
- Text:
Für die letzten zwei Tage dieser langen Reise ...
</user_inst>
```

**`prompt_name`** — the speaker's **name** in place of the reference audio, so the model can
memorise the voice. Byte-identical to `prompt_reference` except the `Reference(s):` slot:

```
- Reference(s):
Speaker: Katrin
```

`- Quality:`, `- Sound Event:` and `- Ambient Sound:` are `None` because the generator never
passed them — that is the literal prompt shape the model saw. **`- Tokens:` is a reconstruction**:
the generator passed a length budget derived from the word count, raised for non-verbal carriers
like screams and groans; the value here is the aligned `n_words` of the same text, which agrees
with it on ordinary lines but not necessarily on the burst-heavy ones. It is the one field of the
prompt that is rebuilt rather than lifted. Everything else in `prompt_reference` is either the
generator's own string (`instruction_generation`) or a corpus column.

Train on either mode, or mix them per step; `speaker_name` and `ref_*` are both present on every row.
`instruction_generation` is the generator's own conditioning string; `instruction_caption` is an
*independently* rendered description of the delivered audio, drawn from the corrected numeric
annotation using one of the **16 procedural templates** (`caption_template` says which — it is
rotated per record so no template dominates). Swap one for the other in the `- Instruction:` slot
to train caption-following rather than direction-following.

---

## Names

Each profile has one unique given name, and an acoustic gender class measured from
**`vn_GEND_reg`** over that profile's own takes (high = masculine, verified: +0.87 with chest
resonance, −0.55 with head resonance across the 500 per-voice means). Pooled over 1,003,974
clips the distribution is bimodal, modes at **1.587** and **4.867**, 50/50 boundary at
**3.2103**; a voice is `male` when >= 85 % of its takes sit above it, `female` when <= 15 % do,
`uncertain` otherwise — **209 / 170 / 121**. Uncertain voices get names that read as unisex in
Germany or the US. 125 German names, 375 English, following `card_language` where it exists.

`names.csv` / `names.json` ship in this repo, and the same mapping is in
`laion/moss-voice-profile-loras-500` (`name` in `manifest.parquet`, `voice.json`, `profile.json`).

The pre-existing `card_gender` label — a VLM's read of the *reference clip* and the voice's design
spec, not of the generated takes — agrees on **89.2 %** of the 379 confidently classified voices
and disagrees on **41**, which `names.csv` flags. The disagreements track speaker-identity failure:
their rate falls from **15.8 %** in the weakest quartile of `frac_ge06` (share of takes reaching
`spk_sim >= 0.60`) to **3.2 %** in the strongest, monotonically. Note also that the `uncertain`
bucket is mostly *measured* ambiguity, not *designed*: only **12 of 121** carry
`card_gender = Androgynous`.

## Column dictionary

The generated, machine-checked version of this table — all 72 columns, plus the full
40-emotion and 114-VoiceNet lists with the dimension each code stands for — is in
**[`COLUMNS.md`](COLUMNS.md)**.

### `sft` columns

| column | type | meaning |
|---|---|---|
| `uid` | `large_string` | utterance id; joins to `laion/laion-voice-profiles-annotated` and to the score index |
| `voice` | `large_string` | voice-profile id (one of 500) |
| `speaker_name` | `large_string` | the assigned unique given name for this voice |
| `speaker_gender` | `large_string` | acoustic gender class: male / female / uncertain (from `vn_GEND_reg`) |
| `shard` | `large_string` | source shard of the corpus (`vprof_base-NNNNN`) |
| `audio_key` | `large_string` | `<gid>.cNNN.mp3` — the key inside the corpus tar |
| `gid` | `large_string` | condition group: `<voice>|<block-code>|<cell>|<lang>`. 842 per voice |
| `cand` | `int64` | candidate index 0..47 within the gid |
| `subset` | `large_string` | `A` (32 takes, own sentences) or `B` (16 takes, the minimal-pair slots) |
| `sub_idx` | `int64` | index within the subset; with `parent_key` and `lang` it identifies the sentence in B |
| `block` | `large_string` | emotion / voicenet / edge / character / burst_isolated / explicit / sports |
| `lang` | `large_string` | `en` or `de` |
| `cond_key` | `large_string` | the (emotion, condition) or (dim, level) cell, without language |
| `parent_key` | `large_string` | the emotion or dimension the cell belongs to; groups the 4 conditions/levels |
| `text_slot` | `large_string` | slot label of the sentence pool, NOT the sentence |
| `emotion` | `large_string` | requested emotion (emotion block only) |
| `condition` | `large_string` | requested acting condition A/B/C/D (emotion block only) |
| `intensity` | `large_string` | `intense` / `moderate` — decoded from `condition` |
| `containment` | `large_string` | `free` / `contained` — decoded from `condition` |
| `dim` | `large_string` | requested VoiceNet dimension (voicenet block only) |
| `level` | `large_string` | requested level: extremely_low / moderately_low / moderately_high / very_high |
| `edge` | `large_string` | edge-case id (edge block only) |
| `character` | `large_string` | character id (character block only) |
| `burst_class` | `large_string` | vocal-burst class the generator asked for, if any |
| `is_holdout` | `bool` | the gid was a validation gid in the LoRA work; **not** excluded here |
| `text` | `large_string` | the prompt text, verbatim |
| `text_with_bursts` | `large_string` | the same text with detected vocal bursts marked inline |
| `n_words` | `int64` | word count; also the `- Tokens:` budget in the prompt |
| `align_status` | `large_string` | forced-alignment status from the re-annotation (`ok` or an error string) |
| `dur_s` | `double` | duration in seconds, from the generator's pre-encode measurement |
| `instruction_generation` | `large_string` | the literal `GENERAL:`/`SCRIPT:` string the generator was conditioned on |
| `instruction_caption` | `large_string` | an independently rendered description of the delivered audio, from the corrected numeric annotation |
| `caption_template` | `large_string` | which of the 16 procedural templates `instruction_caption` uses |
| `spk_sim` | `double` | ECAPA cosine of this take to the PROFILE's reference embedding (generation time) |
| `genuineness` | `double` | generation-time genuineness, 0-6 |
| `blend` | `double` | generation-time vocal-burst blend, 0-10 |
| `emo_strength` | `double` | generation-time emotion strength |
| `wer` | `double` | generation-time WER — shipped, not used |
| `n_genuineness` | `double` | per-voice quantile rank of `genuineness` |
| `n_blend` | `double` | per-voice quantile rank of `blend` |
| `n_emo_strength` | `double` | per-voice quantile rank of `emo_strength` |
| `genuineness_0_6` | `double` | re-annotated genuineness |
| `blend_0_10` | `double` | re-annotated vocal-burst blend |
| `qual_overall` | `double` | re-annotated overall quality head |
| `qual_speech` | `double` | re-annotated speech quality head |
| `qual_background` | `double` | re-annotated background quality head |
| `n_bursts` | `int64` | re-annotated vocal-burst count |
| `burst_labels` | `list<element: string>` | re-annotated vocal-burst labels |
| `top_emotion` | `large_string` | argmax of the 40 re-annotated emotion intensities |
| `top_emotion_value` | `double` | its value |
| `sft_score` | `double` | `n_genuineness + n_blend + 2*n_emo_strength` |
| `sft_rank` | `int64` | 1..3 within the gid |
| `target_frames` | `int64` | MOSS frames of the target |
| `n_vq` | `int64` | codebooks per frame — always 12 |
| `target_codes` | `binary` | target MOSS codes, little-endian uint16, reshape `(target_frames, 12)` |
| `words_json` | `large_string` | MMS_FA word alignment: `[{w, s, e}, ...]`, seconds |
| `ref_cos` | `double` | ECAPA cosine between this sample and its reference clip (>= 0.60 by construction) |
| `ref_same_parent` | `bool` | whether the reference came from the same `parent_key` |
| `ref_uid` | `large_string` | uid of the reference clip |
| `ref_gid` | `large_string` | its gid — always different from `gid` |
| `ref_parent_key` | `large_string` | its parent_key |
| `ref_lang` | `large_string` | its language |
| `ref_block` | `large_string` | its block |
| `ref_emotion` | `large_string` | its emotion, if any — usually different from the target's |
| `ref_dur_s` | `double` | its duration |
| `ref_spk_sim` | `double` | its own ECAPA cosine to the profile reference |
| `ref_frames` | `int64` | its MOSS frames |
| `ref_codes` | `binary` | its MOSS codes, same encoding as `target_codes` |
| `prompt_reference` | `large_string` | complete prompt, reference-audio mode (`<|audio|>` = `ref_codes`) |
| `prompt_name` | `large_string` | complete prompt, name mode (`Speaker: <speaker_name>`, no audio) |
| `emo_vector` | `binary` | the 40 re-annotated emotion intensities, float32 little-endian, alphabetical |
| `vn_reg_vector` | `binary` | the 57 re-annotated VoiceNet regressions, float32 little-endian, alphabetical |

---

## Limitations, honestly

* **No human has listened to any of this in a controlled study.** Every selection criterion here
  is a model output — VoiceNet dimensions, EmoNet intensities, an ECAPA cosine, a genuineness head.
  "Best 3 of 48" means best by those proxies.
* **The `emotion` DPO family is a same-sentence, different-*condition* comparison, not a
  same-prompt resample.** The corpus contains no repeated generations of one prompt, so the
  negative was generated under a different instruction. It is a legitimate "this line, wrong
  reading" negative, and it is not an "identical prompt, worse sample" negative. Anyone expecting
  the latter should know they are not getting it.
* **`vocal_burst` and `blend` in the family-(a) formula are the same physical quantity measured
  twice** (re-annotation vs generation time). The terms are correlated; the ranking is effectively
  ~2x burst blend + 2x target emotion. Kept as specified rather than silently changed.
* **Truncation and continuation negatives are constructed, not observed.** A real model failure
  looks different from a clean prefix or a hard splice. These teach the shape of the failure, not
  its texture. The continuation seam has no crossfade.
* **Speaker identity is weak against the nominal reference, everywhere.** Per-voice mean `spk_sim`
  spans **0.176 to 0.691**, median **0.445**; the median voice has ~20 % of its takes at 0.60 or
  above. 138 voices sit below 0.40. The takes are mutually consistent — which is why every
  voice still yields references at cosine 0.60 — but "sounds like this profile's reference clip"
  is a much weaker claim than "sounds like one consistent speaker". This is inherited from the
  generation run; nothing in this build could fix it. Filter on `spk_sim` / `ref_spk_sim` if voice
  fidelity to the reference matters more than coverage.
* **The name-conditioned prompt inherits that.** For a drifted voice, `Speaker: <name>` will teach
  the drifted voice, not the reference clip.
* **`is_holdout`** marks the `gid`s reserved as the per-voice validation split in the LoRA work.
  They are **included** here — exclude them yourself if you intend to compare against those
  numbers.
* **Forced alignment is imperfect.** `align_status` is `ok` on the large majority but not all rows;
  where alignment failed there are no words, so no truncation pair was emitted for that take.
* **German and English are balanced by construction** (one `gid` per language per cell), but the
  text pools differ in origin and the German is partly translated.
* **Everything here is model output**, including the reference clips. No take is a recording of a
  person. Only the `emolia_*` family traces back to real recorded speech upstream, and that
  upstream dataset is itself CC-BY-4.0.

---

## Reproducing

Build code is in `code/` in this repo: `build_voice.py` (selection, reference assignment,
truncation points), `emit.py` (record assembly), `run_seq.py` / `run_all.sbatch` (500-way fan-out),
`aggregate.py` + `make_cards.py` (the verification below and this card), `namepools.py`,
`quickstart.py`. It is **CPU-only and cost zero GPU-hours** — every model score it consumes
already existed in the corpus. Actual compute is ~11.1 core-hours of work; it was run as
4 booster nodes for about half an hour, which the site bills at 288 core-h per node-hour whether
or not the GPUs are used.

### Verification run on the shipped files

| check | result |
|---|---|
| rows shipped | SFT **1,200,531** · emotion **1,064,594** · truncation **1,186,406** · continuation **1,200,531** |
| every parquet footer opens, no stray temp files | yes, all 2000 |
| files | 500 + 500 + 500 + 500 parquet, no loose files |
| size | 6.3 GB + 19.1 GB |
| `target_frames == round(dur_s * 12.5)` | 95,680 / 95,680 rows |
| code buffer length == `frames * 12` | 1,600 / 1,600 sampled |
| all codes in 0..1023 | 1,600 / 1,600 sampled |
| reference is from another `gid` | 95,680 / 95,680 |
| `ref_cos >= 0.60` | 95,680 / 95,680 |
| last word end <= `dur_s` | 1,600 / 1,600 sampled |
| duplicate `uid` within a voice | 0 |
| distinct speaker names in 40 sampled voices | 40 |
| truncation cut is a whole frame, inside the clip | 23,578 / 23,578 |
| DPO emotion: `text == rejected_text` | 21,118 / 21,118 |
| DPO emotion: condition or level differs | 21,118 / 21,118 |
| DPO emotion: `target_gap >= 0.15` | 21,118 / 21,118 |
| caption gender agrees with the voice's measured class | 40 / 40 voices |
| ...share of captions saying *masculine* for `male` voices | 97.2 % |
| ...share saying *feminine* for `female` voices | 97.6 % |
| DPO continuation: donor from another `gid` | 23,825 / 23,825 |
| DPO continuation: `rejected_frames == chosen + donor` | 23,825 / 23,825 |
| DPO continuation: donor <= 3 s | 23,825 / 23,825 (mumble 16062, near_silent 7507, short_failed_take 256) |

Each check reads the released parquet and re-derives the property from a different quantity than
the writer used — `frames` against `dur_s * 12.5`, code length against the declared frame count,
`ref_gid` against `gid`, `text` against `rejected_text` — rather than re-running the builder.

---

## Licence and attribution

**CC-BY-4.0.** Credit **Christoph Schuhmann** and **LAION**, and the upstream sources listed in
[`laion/laion-voice-profiles-annotated`](https://huggingface.co/datasets/laion/laion-voice-profiles-annotated).
The 500 profiles are synthetic; 385 of them are model re-interpretations whose source identifiers
are deliberately not reconstructible from any release.

```bibtex
@misc{laion_voice_profiles_tts_2026,
  title  = {LAION Voice Profiles — TTS fine-tuning sets (SFT and DPO)},
  author = {Schuhmann, Christoph and {LAION}},
  year   = {2026},
  url    = {https://huggingface.co/datasets/laion/laion-voice-profiles-sft}
}
```