---
title: llm-jp-chatbot-arena-conversations
canonical_url: "https://www.modelscope.cn/datasets/llm-jp/llm-jp-chatbot-arena-conversations"
md_url: "https://www.modelscope.cn/datasets/llm-jp/llm-jp-chatbot-arena-conversations.md"
repository: llm-jp/llm-jp-chatbot-arena-conversations
last_updated: 2026-10-22
license: cc
storage_size: "19 MB"
downloads: 174
stars: 0
---

# llm-jp-chatbot-arena-conversations

> llm-jp-chatbot-arena-conversations - llm-jp 在 ModelScope 开源的数据集。LLM-jp Chatbot Arena Conversations Dataset

llm-jp/llm-jp-chatbot-arena-conversations 是 ModelScope 魔搭社区上的数据集，存储大小 19 MB，采用 cc 许可。

- **Repository**: llm-jp/llm-jp-chatbot-arena-conversations
- **License**: cc
- **Storage size**: 19 MB
- **Downloads**: 174
- **Stars**: 0
- **Last updated**: 2026-10-22

Source: https://www.modelscope.cn/datasets/llm-jp/llm-jp-chatbot-arena-conversations

---

## LLM-jp Chatbot Arena Conversations Dataset

This dataset contains 2,541 conversations with pairwise human preferences, most of which are in Japanese.
The data was collected through the LLM-jp Chatbot Arena, where users compared responses from two different models in a head-to-head format.
Each sample includes a conversation-pair ID, the names of the two models, their conversation transcripts, the user's vote, an anonymized judge ID, a detected language tag, OpenAI moderation API output, and a timestamp.

To ensure a safe public release, we made our best effort to remove all conversations containing personally identifiable information (PII).
User consent was obtained via the "Terms of Use" on the data collection site.
We also provide the output of the OpenAI moderation API to help identify potentially inappropriate content.
However, we have retained conversations flagged as unsafe to support research on safety concerns in real-world LLM use and the effectiveness of moderation systems.

## Collection Periods

Each collection period is stored in a top-level directory named `{yymmdd}-{yymmdd}` and exposed as a separate dataset configuration with a `conversations` split.
The directory names use the first and last dates present in the source logs, in Japan Standard Time (Asia/Tokyo, UTC+09:00).
They describe the source-log coverage rather than the dates of the earliest and latest retained votes.

| Configuration | Source-log period (JST) | Data file | Samples | Models | Judges |
|:--------------|:-----------------------|:----------|--------:|-------:|-------:|
| `250129-250211` | January 29–February 11, 2025 | [conversations.jsonl](250129-250211/conversations.jsonl) | 990 | 10 | 200 |
| `250616-260308` (default) | June 16, 2025–March 8, 2026 | [conversations.jsonl](250616-260308/conversations.jsonl) | 1,551 | 10 | 534 |

> [!NOTE]
> The previously published `conversations.jsonl` has been moved to `250129-250211/` without changes to its contents.

> [!NOTE]
> Model and judge counts are calculated separately for each configuration. Judge IDs are assigned independently within each collection period; the same ID in different configurations does not identify the same judge.

## Loading the Dataset

Select a collection period in the Dataset Viewer, or specify its configuration name when loading with Hugging Face Datasets:

```python
from datasets import load_dataset

repo_id = "llm-jp/llm-jp-chatbot-arena-conversations"
previous = load_dataset(repo_id, "250129-250211", split="conversations")
latest = load_dataset(repo_id, "250616-260308", split="conversations")
```

Loading without a configuration name selects the latest collection period, `250616-260308`.

Both configurations use the schema declared in the YAML metadata above:

- `conversation_a` and `conversation_b` are lists of messages with `content`, `role`, and `speaker` fields.
- `openai_moderation` is a nested object containing `flagged`, `categories`, `category_scores`, and `category_applied_input_types`. The flag is accessed as `openai_moderation.flagged`, not as a top-level column.
- `tstamp` is a Unix timestamp in seconds.

## Disclaimers

- **This dataset includes conversations that may contain sensitive, offensive, or potentially upsetting content.** It is provided to support research on language model behavior, safety, and robustness. When using this dataset for training or evaluation, we strongly encourage the application of appropriate safety measures and content filtering.
- Statements and opinions expressed in the dataset do not represent the views of the researchers or affiliated institutions involved in its creation.

## License

User prompts are licensed under CC BY 4.0, while model outputs are subject to their respective licenses.

## Citation

If you use this dataset, please cite the following paper. The English title and author names are taken from the paper; the main text is in Japanese.

```bibtex
@article{kiyomaru2026llmjpchatbotarena,
  author = {Kiyomaru, Hirokazu and Sugiyama, Hiroaki and Tokoyo, Hiroshi and Kubo, Takahiro and Okazaki, Naoaki},
  title = {{LLM-jp Chatbot Arena}: Building a Dialogue Evaluation Platform for Japanese {LLMs}},
  journal = {Proceedings of the Annual Conference of JSAI},
  volume = {JSAI2026},
  pages = {3YinA10-3YinA10},
  year = {2026},
  note = {In Japanese},
  doi = {10.11517/pjsai.JSAI2026.0_3YinA10},
  url = {https://doi.org/10.11517/pjsai.JSAI2026.0_3YinA10},
}
```