---
title: DuplexJev-32B-Para
canonical_url: "https://www.modelscope.cn/models/AdventistsAI/DuplexJev-32B-Para"
md_url: "https://www.modelscope.cn/models/AdventistsAI/DuplexJev-32B-Para.md"
repository: AdventistsAI/DuplexJev-32B-Para
last_updated: 2026-10-22
license: cc-by-nc-4.0
model_type:
  - ultravox
architectures:
  - DuplexJevForConditionalGeneration
base_model:
  - adventists-ai/DuplexJev-32B-Turn
  - Qwen/Qwen3-VL-32B-Instruct
  - Qwen/Qwen3-ASR-0.6B
base_model_relation: finetune
parameters: 33.0B
tensor_type:
  - BF16
library_name:
  - safetensors
  - pytorch
frameworks:
  - PyTorch
language:
  - zh
  - en
  - ja
  - ko
  - vi
  - th
  - ar
  - ru
downloads: 46
stars: 2
tags:
  - duplexjev
  - speech-to-decision
  - full-duplex
  - paralinguistics
  - nonverbal
  - prosody
  - speaker-verification
  - turn-detection
  - accent
  - multilingual
  - speech-llm
  - vllm
---

# DuplexJev-32B-Para

> DuplexJev-32B-Para - AdventistsAI 在 ModelScope 开源的模型。DuplexJev-32B-Para (v1.4)

AdventistsAI/DuplexJev-32B-Para 是 ModelScope 魔搭社区上的 33.0B 参数机器学习模型，采用 cc-by-nc-4.0 许可，基于 adventists-ai/DuplexJev-32B-Turn、Qwen/Qwen3-VL-32B-Instruct、Qwen/Qwen3-ASR-0.6B 构建。

- **Repository**: AdventistsAI/DuplexJev-32B-Para
- **License**: cc-by-nc-4.0
- **Parameters**: 33.0B
- **Base model**: adventists-ai/DuplexJev-32B-Turn, Qwen/Qwen3-VL-32B-Instruct, Qwen/Qwen3-ASR-0.6B
- **Tags**: duplexjev, speech-to-decision, full-duplex, paralinguistics, nonverbal, prosody, speaker-verification, turn-detection, accent, multilingual, speech-llm, vllm
- **Downloads**: 46
- **Stars**: 2
- **Last updated**: 2026-10-22

Source: https://www.modelscope.cn/models/AdventistsAI/DuplexJev-32B-Para

---

# DuplexJev-32B-Para (v1.4)

📄 Paper: [arXiv:2610.02638](https://arxiv.org/abs/2610.02638) · Code: [github.com/adventists-ai/duplexjev](https://github.com/adventists-ai/duplexjev) · Release notes: [Japanese as a third language](https://github.com/adventists-ai/duplexjev/blob/main/research/2026-10-japanese.md) · [weak languages](https://github.com/adventists-ai/duplexjev/blob/main/research/2026-10-weak-languages.md) · Research notes: [hearing stress](https://github.com/adventists-ai/duplexjev/blob/main/research/2026-10-stress-understanding.md) · [keeping a speech-decision LLM able to talk](https://github.com/adventists-ai/duplexjev/blob/main/research/2026-10-chat-replay.md)

**The DuplexJev model that hears the most beyond the words.** On top of everything
[DuplexJev-32B-Turn](https://huggingface.co/adventists-ai/DuplexJev-32B-Turn) and
[DuplexJev-32B-Stress](https://huggingface.co/adventists-ai/DuplexJev-32B-Stress) do (spoken QA, gender, emotion,
turn-taking, stress and pauses), it also hears **laughter, breathing, coughs and sighs**, notices **how** someone is
talking (whispering, shouting, very fast or slow), and can tell, roughly as a person would, **whether two clips are
the same speaker**. Mandarin and English. Since v1.1 it also **holds a normal conversation**, multi-turn included.
**v1.2** adds **accent judgments** (English accents and Mandarin dialect regions) and much sharper **end-of-turn
detection right after the user pauses**: on LiveKit's eot-bench it now beats Smart Turn v3.2 in English and Chinese.
**v1.3** hears **stress** much better: on the public StressTest benchmark it goes from 72 to **85**, on StressPresso's
meaning questions from 69 to **89**, and on choosing the meaning a stressed word implies (DEBATE, held-out) from 85 to **93**, with the
other decisions about the same.
**v1.4** understands far more languages: **Japanese** error goes from 33 % to **12 %** (FLEURS) and from 38 % to
**11 %** on conversational Japanese, Korean and Vietnamese error falls to a quarter to a third of v1.3, Thai,
Arabic and Russian error by 30–40 %, and Mandarin conversational paralinguistics (laughter, breath, drawl, stutter …) are much sharper.

> **v1.4 (2026-10-22)** replaces v1.3 (2026-10-15): many more languages, Mandarin paralinguistics much better.
> Earlier versions stay available as revisions `v1.3`, `v1.2`, `v1.1` and `v1.0` (`--revision v1.3` in vLLM,
> `revision="v1.3"` in `from_pretrained`).

DuplexJev reads typed, closed-set decisions from speech as a **single-token distribution** of the LLM: zero decode
steps, many questions about the same clip in parallel. This repository is the complete model (Qwen3-ASR-0.6B encoder
+ connector + the language model of Qwen3-VL-32B, 33.0 B parameters, bf16; one 80 GB GPU).

**Which 32B to pick.** This one, for everything above.

## What changed

Starting from DuplexJev-32B-Turn we trained the connector and a new rank-16 LoRA on the LLM (merged here) for 4,000
steps. The mix is the 32B-Turn mix plus:

| added data | what it teaches |
|---|---|
| stress / pause detection (as in 32B-Stress: DEBATE, Stress-17K, TinyStress-15K, Expresso, neutral negatives) | which word is stressed, or none; where the pause is |
| speaker pairs (VoxCeleb1 dev, AISHELL-1, LibriSpeech, two-speaker mixtures) | same speaker or not; is this person in the clip |
| non-verbal sounds (VocalSound, NonVerbalSpeech-38K, NonverbalTTS, Expresso laughing) | is there laughter / breathing / a cough / a sigh / sniffing / throat clearing / a sneeze / crying …; negatives are cut from the same recordings away from the event |
| speaking style (EARS, Expresso whisper) | whispering, loud, fast, slow, high or low pitch, normal |
| spoken multiple-choice replay (ARC, OpenBookQA and CommonsenseQA train splits read by TTS) | keeps "answer the content" from being crowded out by "listen to the voice" |
| **v1.1:** conversation replay (spoken and typed user turns, up to 6 earlier turns as history; target = the original LLM's own reply) and in-house chat items | normal, full-length replies; trained with per-row loss averaging so long replies do not swamp the one-token decisions |
| **v1.2:** accents (VCTK, English Dialects, EdAcc, Common Voice accent labels, KeSpeech; held-out speakers kept for testing) | which accent: 19 English accents and 9 Mandarin dialect regions; native speaker or not; standard Mandarin or not; which English region |
| **v1.2:** end of turn right after a pause (otoSpeech-141h cut 0.2 s into each pause, with DualTurn labels; Smart Turn v3.2 training set) | finished vs. still talking / hesitating when the user has just gone quiet, the way a live voice agent has to decide |
| **v1.3:** the stress data re-balanced (more "which meaning does this stress imply" items, weight raised) and the gender data weighted ×2; no new corpora | choosing the meaning a stressed word implies; keeping gender where it was |
| **v1.4:** weak-language speech (ja, ko, vi, th, ar, ru: Emilia, Emilia-YODAS, YODAS2, Zeroth-Korean, VIVOS, FLEURS train, Common Voice; 162 k, labels re-checked with Qwen3-ASR-0.6B) + continuation (36 k) | transcribe and understand these languages |
| **v1.4:** Japanese speech (Emilia JA, Emilia-YODAS JA, Common Voice ja and other Japanese speech data; 358 k, re-checked) + Japanese continuation (20 k) | Japanese close to Chinese and English |
| **v1.4:** Mandarin conversational paralinguistics (licensed MagicData corpus, training speakers only; 42 k questions) | laughter, laughing while talking, breath, lip smack, sniff, cough, stressed word, drawl, which character is drawn out, stutter, sudden noise, unnatural pause, sentence emotion |
| **v1.4:** more emotion classes (surprise, fear, anxiety, excitement, encouragement, apologetic, playful) from CSEMOTIONS, EmotionTalk, DailyTalk, JL-Corpus, EMNS, Emozionalmente, SUBESCO | emotions beyond the basic four |

v1.4 was trained from v1.3 in four short stages (2,000 + 2,000 + 1,500 + 1,000 steps; connector + a new rank-16 LoRA
each time, merged here), replaying the full v1.3 mix in every stage; the last two stages raise the stress,
non-verbal, emotion and speaking-style replay ×1.5 each.

## v1.4 vs v1.3

Languages (FLEURS test, 150 utterances per language, median word / character error, lower is better):

| | ja | ko | vi | th | ar | ru | zh | en | es | fr | de |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| v1.3 | 32.7 | 31.8 | 41.7 | 29.1 | 27.0 | 10.6 | 3.8 | 1.5 | 4.4 | 8.5 | 5.3 |
| **v1.4** | **11.6** | **7.6** | **14.6** | **18.0** | **19.4** | **6.9** | 3.8 | 0.0 | 3.5 | 7.0 | 5.3 |
| encoder ceiling (Qwen3-ASR-0.6B) | 7.0 | 2.7 | 7.8 | 5.1 | 16.7 | 7.8 | 2.0 | 1.3 | 3.4 | 6.2 | 5.6 |

Japanese, conversational (held-out Emilia JA shard) / Common Voice ja test, character error: v1.3 38.2 / 33.3 →
**v1.4 11.1 / 13.0** (encoder 8.3 / 10.8).

Mandarin conversational paralinguistics (licensed MagicData corpus, held-out speakers, accuracy %; yes/no unless noted):

| | breath | lip smack | laughter | laughing while talking | sniff | cough | drawl | stressed word (4-way) | emotion (4-way) |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| **v1.4** | 91.7 | 85.8 | 89.3 | 87.8 | 87.5 | 71.4 | 79.4 | 85.3 | 70.7 |

(v1.3 was not trained on this corpus; the 4B-Para v1.1, also untrained on it, scores 50–66 on the yes/no rows.)

Accent, same-speaker and turn-taking (eot-bench, Smart Turn) were **not re-run** for v1.4; the numbers in the next
sections are v1.3's.

## Results (v1.3 and earlier)

| | 32B-Turn | 32B-Para v1.1 | 32B-Para v1.2 | **32B-Para v1.3** |
|---|---:|---:|---:|---:|
| qa100 / ZJU-ML | 96 / 87 | 96 / 88 | 95 / 87 | 96 / 87 |
| gender / emotion | 91.5 / 91.1 | 92.9 / 91.5 | 93.9 / 90.1 | 93.8 / 92.1 |
| VoiceBench OBQA‡ / MMSU | 85.5 / 72.1 | 94.5 / 75.5 | 94.3 / 75.0 | 94.1 / 75.7 |
| Easy-Turn* | 95.2 | 94.8 | **95.5** | 94.4 |
| which word is stressed, "none" offered: DEBATE (zh) / StressPresso (en) | 35 / 1 | 93 / 96 | 92 / 91 | **95 / 96** |
| neutral speech → "no word is stressed": AISHELL-1 / LibriSpeech | 92 / 50 | 100 / 99 | 99 / — | 98 / 83 |
| **meaning from stress: DEBATE held-out / StressTest** | 35 / 52 | 85 / 76 | 85 / 72 | **93 / 85** |
| StressPresso, which meaning the stress implies (English) | — | — | 69 | **89** |
| **laughter / breathing / cough / sigh** (AUC, NonverbalTTS test, real speech, unseen) | — | 0.88 / 0.81 / 0.85 / 0.80 | 0.87 / 0.81 / 0.84 / 0.75 | ≈ v1.2† |
| laughter / breathing in natural Mandarin conversation (AUC, unseen third-party sample) | — | 0.83 / 0.65 | 0.80 / 0.56 | ≈ v1.2† |
| **speaking style, 7 classes** (EARS held-out speakers) | — | 82% | 79% | 80% |
| same speaker? EER, VoxCeleb1 random / hard pairs (our harness) | — | 28.2% / 39.6% | 31.2% / 42.3% | 29.1% / 38.4%◊ |
| **accent, 4 choices**: English (19 accents) / Mandarin (9 dialect regions)§ | — | 39.6 / 25.7 | **56.0 / 48.2** | — |
| **native English speaker? / standard Mandarin?**§ | — | 84.8 / 46.6 | **91.1 / 91.3** | — |
| **LISTEN** (EACL 2026), mean of the 4 audio-only conditions, full set | — | — | 38.4 | **38.9** |

Paper protocol (single-token readout, PyTorch), % correct unless marked; AUC 0.5 = chance.
Through vLLM with `duplexjev-vllm` in the client wording, v1.3: qa100 97, ZJU-ML 90, gender 92.1, emotion 91.8,
Easy-Turn 94.5 (v1.2: 98, 90, 93.6, 89.8, 95.5; v1.1: 97, 91, 93.2, 90.0, 94.8; qa100 and ZJU-ML have 100 items each).
"—": not re-run for that version.

- † v1.3 non-verbal sounds were checked by accuracy on the same test sets (NonverbalTTS, NonVerbalSpeech-38K,
  VocalSound, Expresso, MagicData): within a few points of v1.2 either way; AUC not re-run.
- ◊ Same harness re-run for v1.2: 30.7% / 38.2% (the v1.2 column is from an earlier run).
- Accents for v1.3 were checked only on the cross-corpus set: standard Mandarin or not 88.8 (v1.2 93.1, 160 items),
  Mandarin dialect 4-way 61.3 (60.0); English accents stay near chance cross-corpus for both versions.

- \* Easy-Turn is in-domain (its training split is in the mix).
- § Accent test items come from the same corpora as the training items (different speakers); chance is 25% / 50%.
- ‡ OpenBookQA's train split is in the replay data, so the OBQA gain is partly in-domain; MMSU is unrelated to it.
- For reference, an AudioSet AST tagger scores 0.71 / 0.75 / 0.85 / 0.76 on the same four sounds.
- Speaker: a dedicated speaker-verification model reaches ~1% EER on VoxCeleb1; this model is meant to have a rough,
  human-like sense of "same voice or not", not to identify people.

## Turn-taking benchmarks (v1.2 / v1.3; not re-run for v1.4)

**LiveKit [eot-bench](https://github.com/livekit/eot-bench)** (validation set, decision 0.2 s into each pause, official
metrics). False cut-offs when the agent may wait at most 300 / 600 ms (lower is better):

| | English | Chinese |
|---|---:|---:|
| LiveKit Turn Detector v1 | 9.9 / 4.5% | 20.6 / 8.7% |
| **DuplexJev-32B-Para v1.2, with the assistant's previous message** (plugin default, full eot-bench run) | **20.0 / 9.4%** | **37.7 / 12.2%** |
| DuplexJev-32B-Para v1.2, with the assistant's previous message, `readout="act"` (fast mode) | 18.4 / 7.9% | 37.3 / 12.3% |
| DuplexJev-32B-Para v1.2, audio only (fast mode) | 23.5 / 11.1% | 40.0 / 12.8% |
| **DuplexJev-32B-Para v1.3, with the assistant's previous message** (fast mode) | 20.3 / 8.8% | **35.5** / 12.3% |
| DuplexJev-32B-Para v1.3, audio only (fast mode) | 24.7 / 11.2% | 38.8 / 12.0% |
| DuplexJev-32B-Turn, audio only (fast mode) | 29.8 / 13.9% | 44.9 / 14.5% |
| Smart Turn v3.2 | 35.2 / 14.8% | 42.9 / 13.2% |

eot-bench audio is not in our training data. "With the assistant's previous message" puts
`The assistant had just said: "..."` before the audio; the Pipecat and LiveKit plugins send it
(`pipecat-duplexjev>=0.3`, `livekit-plugins-duplexjev>=0.2`, gateway `duplexjev>=0.6`). Plugin default: the turn
question, P(finished) + P(backchannel); run with every eot-bench decision point (adapter
`benchmarks/turn_detection/eot_bench/duplexjev_ctx_adapter.py` on GitHub). `readout="act"` (`pipecat-duplexjev>=0.4`,
`livekit-plugins-duplexjev>=0.3`): question "What should the assistant do now?", P(reply now) + P(short
acknowledgement). Rows marked fast mode ask once per pause, at the 0.2 s score point.

**Pipecat [Smart Turn v3.2](https://huggingface.co/datasets/pipecat-ai/smart-turn-data-v3.2-test) test set**
(accuracy at threshold 0.5): v1.2 English 94.0% (Smart Turn v3.2: 94.7%), Chinese 91.1% (90.5%); v1.3 94.2% / 91.4%; DuplexJev-32B-Turn
65.2% / 84.6%. The Smart Turn v3.2 *training* set is in our training mix, so this is an in-distribution result.

## Serve with vLLM

```bash
pip install "vllm[audio]>=0.29" duplexjev-vllm
vllm serve adventists-ai/DuplexJev-32B-Para --max-model-len 4096
```

Prompt format and client are the same as for [DuplexJev-32B-Turn](https://huggingface.co/adventists-ai/DuplexJev-32B-Turn);
see [`examples/vllm_client.py`](https://github.com/adventists-ai/duplexjev/blob/main/examples/vllm_client.py). Examples:

```
The user said: <|audio|>

Is there laughter in this audio?

Options:
A. Yes
B. No

Answer with the letter only.
```

```
用户说：<|audio|>

除了说话，这段音频里还有哪种声音？

选项：
A. 叹气
B. 咳嗽
C. 笑声
D. 都没有

请只回答选项字母。
```

Always offer a "none" option for stress and non-verbal questions; the model was trained with it.

## Limitations

- Turn-taking, accent and same-speaker scores were not re-measured for v1.4.
- Thai and Arabic are still 2–3× the encoder's own error.
- Non-verbal training audio is mostly acted (radio drama, animation, studio recordings); in natural conversation the
  scores are lower (see the table).
- Breathing in natural Mandarin conversation is weak (AUC 0.56 in v1.2, 0.65 in v1.1, 0.78 in v1.0); acted / read
  speech is much better.
- Accent answers describe how someone sounds, not where they are from or who they are; accents are often mixed or
  ambiguous. Do not use them to make decisions about people.
- Asked about itself, it may describe abilities (seeing images, speaking) that this checkpoint does not have on its
  own: it takes speech and text and replies in text.
- Speaker judgments are coarse (EER 18–40%) and must not be used to identify or verify people.
- Emotion labels come from acted corpora; do not use the outputs to make decisions about individuals.

## License

**CC BY-NC 4.0, non-commercial use only.**

- Non-commercial data:
  - Stress: DEBATE (github.com/SmileHnu/DEBATE), Stress-17K and TinyStress-15K (slprl);
  - Expresso (Meta), EARS (Meta) and NonVerbalSpeech-38K, all CC BY-NC 4.0;
  - ESD (research only); SmoothConv (CC BY-NC).
  - Accents (v1.2): KeSpeech (non-commercial research use).
  - Conversation replay (v1.1): user turns from wilzzzz/paralinguistic_dialogues (no licence stated on the dataset card; used for non-commercial research), SmoothConv (CC BY-NC), AISHELL-1 (Apache-2.0), LibriSpeech (CC BY 4.0); target replies generated by the base LLM (Apache-2.0); spoken chat questions synthesised with CosyVoice2 (Apache-2.0).
- Added in v1.4 (non-commercial or share-alike): Emilia JA/KO (CC BY-NC 4.0); VIVOS (CC BY-NC-SA 4.0); EmotionTalk
  (CC BY-NC-SA 4.0); DailyTalk (CC BY-SA 4.0); a licensed Mandarin conversational corpus with paralinguistic annotation
  (MagicData, purchased licence; audio not redistributed); other Japanese speech data used for non-commercial research.
- Added in v1.4 (other): Emilia-YODAS JA/KO (CC BY 4.0), YODAS2 (CC BY 3.0), Zeroth-Korean (CC BY 4.0), FLEURS train split
  (CC BY 4.0; the test split was never used), Common Voice 17 ja/ar/ru (CC0), CSEMOTIONS (Apache-2.0), EMNS (Apache-2.0),
  JL-Corpus (CC0), Emozionalmente (CC BY 4.0), SUBESCO (CC BY 4.0). Labels were re-checked, and for unlabelled YODAS2
  clips produced, with Qwen3-ASR-0.6B (Apache-2.0); continuation targets were written by Qwen3-32B (Apache-2.0).
- Other data:
  - VocalSound (CC BY-SA 4.0), NonverbalTTS (audio from VoxCeleb and Expresso);
  - VoxCeleb1 (CC BY 4.0), AISHELL-1 (Apache-2.0), LibriSpeech (CC BY 4.0);
  - ARC (CC BY-SA 4.0), OpenBookQA, CommonsenseQA (MIT).
  - Accents (v1.2): VCTK (CC BY 4.0), English Dialects (CC BY-SA 4.0), EdAcc (CC BY-SA 4.0), common-accent from
    Common Voice (CC0 audio).
  - Turn-taking (v1.2): otoSpeech-141h (CC BY 4.0) with DualTurn labels; Smart Turn v3.2 training set
    (pipecat-ai, CC BY 4.0).
- Base models keep their licences: Qwen3-VL-32B-Instruct and Qwen3-ASR-0.6B are Apache-2.0 (Alibaba Cloud).
- The LLM weights are the language model of Qwen3-VL-32B-Instruct with LoRAs merged in; the vision encoder is not
  included.

## Citation

```bibtex
@misc{jin2026duplexjev,
  title         = {Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen {LLM} Hear Beyond the Transcript},
  author        = {Jin, Jie and Ma, Ziyin and Yin, Min and Chen, Jinyu and Song, Haigang and Pang, Zhikun and Zhang, Xiaowen},
  year          = {2026},
  eprint        = {2610.02638},
  archivePrefix = {arXiv},
  note          = {Submitted to IEEE ICASSP 2027}
}
```

Built by [Adventists.ai](https://github.com/adventists-ai/duplexjev). Claude (Anthropic) assisted with code.