---
title: Speaker-Embedding-30M-Low-Quality-Speech
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/Speaker-Embedding-30M-Low-Quality-Speech"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/Speaker-Embedding-30M-Low-Quality-Speech.md"
repository: AI-ModelScope/Speaker-Embedding-30M-Low-Quality-Speech
last_updated: 2026-09-26
license: cc-by-nc-4.0
library_name:
  - pytorch
  - onnx
frameworks:
  - pytorch
language:
  - vi
downloads: 5
stars: 1
tags:
  - speaker-verification
  - speaker-embedding
  - speaker-recognition
  - onnx
---

# Speaker-Embedding-30M-Low-Quality-Speech

> Speaker-Embedding-30M-Low-Quality-Speech - AI-ModelScope 在 ModelScope 开源的模型。Vietnamese Speaker Embedding — ResNet293 / ResNet34 for Low-Quality Speech (ONNX)

AI-ModelScope/Speaker-Embedding-30M-Low-Quality-Speech 是 ModelScope 魔搭社区上的机器学习模型，采用 cc-by-nc-4.0 许可。

- **Repository**: AI-ModelScope/Speaker-Embedding-30M-Low-Quality-Speech
- **License**: cc-by-nc-4.0
- **Tags**: speaker-verification, speaker-embedding, speaker-recognition, onnx
- **Downloads**: 5
- **Stars**: 1
- **Last updated**: 2026-09-26

Source: https://www.modelscope.cn/models/AI-ModelScope/Speaker-Embedding-30M-Low-Quality-Speech

---

# Vietnamese Speaker Embedding — ResNet293 / ResNet34 for Low-Quality Speech (ONNX)

## 🔍 Overview
Two **speaker-embedding** models for **Vietnamese low-quality speech**, exported to **ONNX** and ready for CPU or GPU inference with `onnxruntime`:

| Model | Parameters | File size | Embedding |
|-------|-----------|-----------|-----------|
| **ResNet293** (higher accuracy) | **28.6M** | 114 MB | 256-d |
| **ResNet34** (faster) | **6.6M** | 27 MB | 256-d |

Given any audio clip, the model returns a 256-dimensional voiceprint. Two clips are compared with cosine similarity — the higher the score, the more likely they come from the same speaker. Typical uses: **speaker verification**, **voice-based login**, **speaker search / de-duplication** in voice-bot systems handling low-quality speech.

On simulated Vietnamese low-quality speech, ResNet293 reaches **11.2% EER** on a balanced 29k-trial list — about **2.3× lower error** than general-purpose models trained on VoxCeleb / CN-Celeb (25–27%).

---

## ☕ Give Me a Coffee for Training Cost

This model was trained entirely on **self-funded, personal hardware** — every GPU hour and electricity bill came out of my own pocket.  
If this work is useful to you, a small contribution helps cover the cost of the next training run and keeps future models free and open. Every coffee is hugely appreciated 🙏

<p align="center">
  <img src="qr.jpg" alt="VietQR — Give me a coffee" width="320"/>
</p>

<p align="center"><i>Scan with any VietQR / Napas 247 supported banking app</i></p>

---

## ⚙️ Model Architecture

- **Backbone:** ResNet (WeSpeaker-style), 2-D convolutions over the log-mel spectrogram
  - **ResNet293:** bottleneck blocks `[10, 20, 64, 3]`, base width 32 (stage outputs 128 / 256 / 512 / 1024 channels)
  - **ResNet34:** basic blocks `[3, 4, 6, 3]`, base width 32
- **Pooling:** temporal statistics pooling (mean + standard deviation over time)
- **Projection:** fully-connected layer → **256-d** embedding
- **Export:** ONNX opset 14, BatchNorm folded into convolutions, dynamic batch and length axes

| | Input | Output |
|---|---|---|
| Name | `feats` | `embs` |
| Shape | `[batch, frames, 80]` | `[batch, 256]` |

**Front-end (done outside the model):** 16 kHz mono → 80-dim Kaldi fbank (25 ms window, 10 ms shift) computed on **int16-scale** samples → cepstral mean normalisation (subtract the per-utterance mean). Embeddings should be L2-normalised before scoring.

> ⚠️ The fbank must be computed on int16-scale samples (as `torchaudio.load(normalize=False)` returns), **not** on floats in `[-1, 1]` — otherwise every embedding shifts.

---

## 🧠 Training Data

The models were trained on a mix of Vietnamese and English speaker data:

| **Dataset** | **Language** | **How it is used** |
|-------------|--------------|--------------------|
| [PhoAudiobook](https://huggingface.co/datasets/thivux/phoaudiobook) | Vietnamese | ~1,000 hours of speaker-labelled audiobook speech |
| [viVoice](https://huggingface.co/datasets/capleaf/viVoice) | Vietnamese | Pseudo-labelled into per-speaker sets: utterances are embedded with a pretrained **WavLM** speaker model and clustered with **HDBSCAN**; duplicate-speaker checks then merge clusters that belong to the same voice, so one speaker is never split across several labels |
| [VoxCeleb1](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox1.html) | Multilingual (mostly English) | Speaker-labelled celebrity interviews |
| [VoxCeleb2](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox2.html) | Multilingual (mostly English) | Speaker-labelled celebrity interviews |

<!-- TODO: training recipe (loss / margin, augmentation, epochs) and total #speakers / hours. -->

---

## 🧪 Evaluation Results

All numbers below are **utterance-vs-utterance trials**, scored with cosine similarity, no score normalisation.  
Metrics: **EER** (%) and **minDCF** (P_target = 0.01). *Lower is better.*

### Vietnamese low-quality speech synthesised by OmniVoice
Simulated 16 kHz low-quality speech recordings, 150 speakers, 3,783 utterances (median 2.1 s).  
Balanced trial list: **29,162 trials** (50% target / 50% non-target, up to 100 target trials per speaker).

| **Model** | **EER (%)** | **minDCF** |
|-----------|-------------|------------|
| **ResNet293 (this)** | **11.20** | **0.418** |
| **ResNet34 (this)** | 15.15 | 0.557 |
| WeSpeaker VoxBlink2 SimAM-ResNet100 | 24.66 | 0.853 |
| WeSpeaker CN-Celeb ResNet34 | 24.68 | 0.927 |
| ReDimNet2-B6 (VoxBlink2+Vox2+CNC) | 25.16 | 0.859 |
| WeSpeaker VoxBlink2 SimAM-ResNet34 | 25.19 | 0.832 |
| ReDimNet2-B3 (VoxBlink2+Vox2+CNC) | 26.55 | 0.892 |
| WeSpeaker VoxCeleb ResNet34 | 27.42 | 0.926 |

### VoxVietnam (test split)
[`hustep-lab/VoxVietnam-Dataset`](https://huggingface.co/datasets/hustep-lab/VoxVietnam-Dataset), multi-genre Vietnamese speech. Balanced sample of 50 utterances per speaker (150 speakers, 4,417 utterances, cropped to 20 s); **22,048 trials** (50/50).  
*Simulated low-quality speech* = degraded channel: 300–3400 Hz band-limiting, line noise (SNR 10–30 dB), G.711 µ-law compression, 0–5% bursty packet loss.

| **Model** | **Clean** | **Simulated low-quality speech** |
|-----------|-----------|----------------------------------|
| **ResNet293 (this)** | 12.45 | **12.94** |
| **ResNet34 (this)** | 12.85 | 14.60 |
| WeSpeaker CN-Celeb ResNet34 | **11.41** | 14.67 |
| ReDimNet2-B3 | 12.83 | 14.41 |
| ReDimNet2-B6 | 13.32 | 13.80 |
| WeSpeaker VoxBlink2 SimAM-ResNet34 | 11.82 | 13.45 |
| WeSpeaker VoxBlink2 SimAM-ResNet100 | 11.96 | 13.81 |
| WeSpeaker VoxCeleb ResNet34 | 13.73 | 16.25 |

> EER (%). The VoxVietnam parquet release has no official trial list (VoxVietnam-E/H) and no session metadata, so these trials are sampled and may include same-video pairs — use them to compare models, not as absolute error rates.

### VoxCeleb1-O (English, official cleaned trial list, 37,611 trials)

| **Model** | **Clean** | **Simulated low-quality speech** |
|-----------|-----------|----------------------------------|
| **ResNet293 (this)** | 0.64 | 5.01 |
| **ResNet34 (this)** | 1.43 | 9.07 |
| WeSpeaker VoxCeleb ResNet34 | 0.92 | 3.88 |
| ReDimNet2-B3 | 0.51 | 2.58 |
| ReDimNet2-B6 | **0.25** | **1.62** |
| WeSpeaker VoxBlink2 SimAM-ResNet34 | 1.19 | 5.31 |
| WeSpeaker VoxBlink2 SimAM-ResNet100 | 0.83 | 4.39 |

> The models are specialised for Vietnamese low-quality speech; on English — clean or degraded — large general-purpose models such as ReDimNet2 remain stronger.

---

## 🎯 Recommended Thresholds

Scores below use the `(cos + 1) / 2` mapping (range 0–1). Thresholds were calibrated on the in-house low-quality speech set with a **5-utterance enrollment centroid** (average of 5 L2-normalised embeddings, re-normalised), scoring held-out utterances with ≥ 1.5 s of speech:

| **Model** | **Accept ≥** | **Reject <** | **EER (centroid protocol)** |
|-----------|--------------|--------------|-----------------------------|
| ResNet293 | 0.75 | 0.64 | 4.3% |
| ResNet34 | 0.74 | 0.60 | 5.7% |

Scores between the two thresholds are *inconclusive* (ask for more audio). Re-calibrate on your own data if your channel differs.

---

## ⚡ Inference Speed

Measured with `onnxruntime` on CPU (1 thread per process, i5 12600K), including fbank extraction:

| **Model** | **Short utterance (~2 s)** |
|-----------|----------------------------|
| ResNet34 | ~50 ms |
| ResNet293 | ~240 ms |

For real-time use of ResNet293, run on GPU (`onnxruntime-gpu`, `CUDAExecutionProvider`).

<!-- TODO: add measured GPU latency (e.g. RTX 3090, 10 s audio). -->

---

## ⚙️ How to Run This Model

Requirements: `numpy`, `soundfile`, `torch`, `torchaudio`, `onnxruntime` (+ `ffmpeg` for m4a/mp4/webm/amr input).

`config.json` describes both models — file, architecture, ONNX input/output names, the front-end parameters and the decision thresholds — so the code below reads everything from it instead of hard-coding:

```python
import json
import numpy as np
import onnxruntime as ort
import soundfile as sf
import torch
import torchaudio.compliance.kaldi as kaldi
import torchaudio.functional as AF

cfg = json.load(open("config.json"))
fe = cfg["frontend"]
model = cfg["models"]["resnet293"]                # or "resnet34"
arch, io = model["architecture"], model["onnx"]
print(f"{arch['backbone']}: {arch['block']} blocks {arch['num_blocks']}, "
      f"{model['num_parameters']/1e6:.1f}M params, {arch['embedding_dim']}-d embedding")

sess = ort.InferenceSession(model["file"], providers=["CPUExecutionProvider"])

def embed(path):
    data, sr = sf.read(path, dtype=fe["sample_scale"], always_2d=True)   # int16 scale, [T, C]
    pcm = data.astype(np.float32).mean(axis=1)                           # mono (downmix = mean)
    x = torch.from_numpy(pcm)
    if sr != fe["sample_rate"]:
        x = AF.resample(x, sr, fe["sample_rate"]).round()
    feat = kaldi.fbank(x[None], num_mel_bins=fe["num_mel_bins"],
                       frame_length=fe["frame_length_ms"], frame_shift=fe["frame_shift_ms"],
                       dither=fe["dither"], sample_frequency=fe["sample_rate"])
    if fe["cmn"]:
        feat = feat - feat.mean(dim=0)                                   # cepstral mean normalisation
    e = sess.run([io["output"]["name"]], {io["input"]["name"]: feat[None].numpy()})[0][0]
    return e / np.linalg.norm(e)                                         # L2

a, b = embed("speaker_a.wav"), embed("speaker_b.wav")
score = (float(a @ b) + 1) / 2                                           # (cos + 1) / 2
t = model["thresholds"]
verdict = "same speaker" if score >= t["accept"] else "different speaker" if score < t["reject"] else "inconclusive"
print(f"score {score:.4f} -> {verdict}")
```

A ready-made CLI that also handles any audio format (via ffmpeg) is provided as `extract_embedding.py`:

```bash
python extract_embedding.py speaker_embedding_resnet293.onnx a.m4a b.mp3   # prints embeddings + similarity
```

---

## 💬 Summary
The **ResNet293 / ResNet34** speaker-embedding models are built for the audio Vietnamese voice products actually receive: short, noisy, codec-compressed low-quality speech. On simulated low-quality speech recordings they cut verification error by more than half versus strong general-purpose models, while ResNet34 stays light enough for **CPU-only deployment** and ResNet293 offers the best accuracy when a GPU is available.

---

## 🙏 Acknowledgements

Architecture, front-end and ONNX export follow [WeSpeaker](https://github.com/wenet-e2e/wespeaker) (Apache-2.0).  
<!-- TODO: add upstream pretrained-model credits and their license terms if these models were fine-tuned from public checkpoints. -->