---
title: HeartMuLa-Benchmark
canonical_url: "https://www.modelscope.cn/datasets/HeartMuLa/HeartMuLa-Benchmark"
md_url: "https://www.modelscope.cn/datasets/HeartMuLa/HeartMuLa-Benchmark.md"
repository: HeartMuLa/HeartMuLa-Benchmark
last_updated: 2026-01-20
license: "Apache License 2.0"
storage_size: "79 KB"
downloads: 6091
stars: 2
---

# HeartMuLa-Benchmark

> HeartMuLa-Benchmark - HeartMuLa 在 ModelScope 开源的数据集。Overview This benchmark is designed to evaluate music generation models, specifically for use in our HeartMuLa project. It provides a standardized dataset and evaluation metrics to quantify model…

HeartMuLa/HeartMuLa-Benchmark 是 ModelScope 魔搭社区上的数据集，存储大小 79 KB，采用 Apache License 2.0 许可。

- **Repository**: HeartMuLa/HeartMuLa-Benchmark
- **License**: Apache License 2.0
- **Storage size**: 79 KB
- **Downloads**: 6091
- **Stars**: 2
- **Last updated**: 2026-01-20

Source: https://www.modelscope.cn/datasets/HeartMuLa/HeartMuLa-Benchmark

---

# HeartMuLa-Benchmark


## Overview
This benchmark is designed to evaluate music generation models, specifically for use in our **HeartMuLa** project.  
It provides a standardized dataset and evaluation metrics to quantify model performance on multi-language, multi-label music generation tasks.

## Dataset
The dataset contains samples in five languages:

- Chinese
- English
- Japanese
- Korean
- Spanish

Each language folder contains multiple subfolders, each including:

- `tags.txt`: AI-generated music tags, which can be used as conditional input for music generation models.  
- `lyrics.txt`: AI-generated lyrics, also usable as model input.

These files serve as standardized conditions for model evaluation.

## Evaluation

### Lyrics Evaluation
1. Use **HeartTranscriptor** (Link: https://modelscope.cn/models/HeartMuLa/HeartTranscriptor-oss) to transcribe the model-generated music into lyrics.
2. Compute **Word Error Rate (WER)** or **Phoneme Error Rate (PER)** to measure alignment between the generated music and target lyrics.

### Tag Evaluation
1. Use Tencent’s **MuQ-MuLan** to extract embeddings for both the generated music and the reference tags.
2. Compute **Cosine Similarity** to measure semantic alignment between the generated music and target tags.
