FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning
Production-ready voice AI solutions powered by Chroma | Open-source model for developers & researchers
[](https://www.flashlabs.ai/flashai-voice-agents) [](https://huggingface.co/FlashLabs/Chroma-4B) [](https://arxiv.org/abs/2601.11141) [](https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma) [](https://x.com/flashlabsdotai) [](https://www.linkedin.com/company/flashlabs-ai/)Watch our model in action
## Capabilities
Chroma 1.0 is capable of:
- **Speech Understanding:** Processing user audio input directly.
- **Multimodal Generation:** Generating coherent text and speech responses simultaneously.
- **Voice Cloning:** utilizing reference audio prompts to guide speech generation style.
## Usage
### Installation
Ensure you have the necessary dependencies installed. You may need the latest versions of `transformers` and `torch`.
```bash
pip install transformers torch
```
### Loading the Model
```python
import torch
from transformers import AutoModelForCausalLM, AutoProcessor
model_id = "FlashLabs/Chroma-4B" # Or local path
# Load model
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
device_map="auto"
)
# Load processor
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
```
### Inference Example
Here is how to perform a simple conversation with audio input and audio output:
```python
import torch
from IPython.display import Audio
# Construct conversation history
system_prompt = (
"You are Chroma, an advanced virtual human created by the FlashLabs. "
"You possess the ability to understand auditory inputs and generate both text and speech."
)
conversation = [[
{
"role": "system",
"content": [
{"type": "text", "text": system_prompt}
],
},
{
"role": "user",
"content": [
# Input audio file path
{"type": "audio", "audio": "assets/make_taco.wav"},
],
},
]]
# Provide reference audio/text for style or context
prompt_text = ["War and bloodshed throughout the world."]
prompt_audio = ["assets/reference_audio.wav"]
# Process inputs
inputs = processor(
conversation,
add_generation_prompt=True,
tokenize=False,
prompt_audio=prompt_audio,
prompt_text=prompt_text
)
# Move inputs to device
device = model.device
inputs = {k: v.to(device) for k, v in inputs.items()}
# 2. Generate
output = model.generate(
**inputs,
max_new_tokens=100,
do_sample=True,
temperature=0.7,
top_p=0.9,
use_cache=True
)
# 3. Decode Audio
# The model outputs raw tokens; we decode the audio part using the codec
audio_values = model.codec_model.decode(output.permute(0, 2, 1)).audio_values
# Save or play audio (e.g., in Jupyter)
Audio(audio_values[0].cpu().detach().numpy(), rate=24_000)
```
### Citation
If you use Chroma in your research, please cite:
```bibtex
@misc{chen2026flashlabschroma10realtime,
title={FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning},
author={Tanyu Chen and Tairan Chen and Kai Shen and Zhenghua Bao and Zhihui Zhang and Man Yuan and Yi Shi},
year={2026},
eprint={2601.11141},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2601.11141},
}
```
### Contact
For questions or issues, please contact: chroma@flashlabs.ai