---
title: ViDoc2BotRerank
canonical_url: "https://www.modelscope.cn/datasets/DAMO_ConvAI/ViDoc2BotRerank"
md_url: "https://www.modelscope.cn/datasets/DAMO_ConvAI/ViDoc2BotRerank.md"
repository: DAMO_ConvAI/ViDoc2BotRerank
chinese_name: "越南语对话重排数据集"
last_updated: 2024-09-02
license: "Apache License 2.0"
storage_size: "28 MB"
domain:
  - configs
  - text
tasks:
  - "{config_name=default, data_files=[{path=Vi_finetune.csv, split=train}]}"
  - conversational
size_scale:
  - 1k-10k
type:
  - kg-qa
downloads: 5644
stars: 2
---

# ViDoc2BotRerank

> ViDoc2BotRerank - DAMO_ConvAI 在 ModelScope 开源的数据集。Documen-grounded dialogue Goal-oriented document-grounded dialogue systems enable end users to interactively query about domain-specific information based on the given documents. The tasks of querying…

DAMO_ConvAI/ViDoc2BotRerank 是 ModelScope 魔搭社区上的{config_name=default, data_files=[{path=Vi_finetune.csv, split=train}]}、conversational数据集，涉及 configs、text 领域，存储大小 28 MB，采用 Apache License 2.0 许可。

- **Repository**: DAMO_ConvAI/ViDoc2BotRerank
- **License**: Apache License 2.0
- **Tasks**: {config_name=default, data_files=[{path=Vi_finetune.csv, split=train}]}, conversational
- **Domain**: configs, text
- **Storage size**: 28 MB
- **Downloads**: 5644
- **Stars**: 2
- **Last updated**: 2024-09-02

Source: https://www.modelscope.cn/datasets/DAMO_ConvAI/ViDoc2BotRerank

---

### Clone with HTTP
```bash
git clone https://www.modelscope.cn/datasets/AronXiang/ViDoc2BotRetrieval.git
```

 # Documen-grounded dialogue 
Goal-oriented document-grounded dialogue systems enable end users to interactively query about
domain-specific information based on the given documents. The tasks of querying document knowledge via conversational systems continue to attract a lot of attention from both research and industrial communities for various applications. The previous works addressed the task of English and Chinese document-grounded dialogue systems, leaving other languages less well explored. Thus, large communities of users are prevented access to automated services and information. We aim to extend the effort by introducing the Third  ACL DialDoc Workshop shared task involving documents and dialogues in diverse languages. We present this multilingual DGD challenge to encourage researchers to explore effective solutions for (1) transferring a DGD model from a high-resource language to a low-resource language; (2) developing a DGD model that is capable of providing multilingual responses given multilingual documents.


### Description



Specifically，we provide 797 dialogues in Vietnamese (3,446 turns), 816 dialogues in French (3,510 turns), and a corpus of  17272 paragraphs, where each dialogue turn is grounded in a paragraph from the corpus.
We also organize the currently available Chinese and English document-grounded dialogue data. We hope that participants can leverage the linguistic similarities, for example, a large number of Vietnamese words are derived from Chinese, and English and French both belong to the Indo-European language family, to improve their models' performance in Vietnamese and French.
So the task objective is to rerank relevant paragraphs from a corpus based on the dialogue history and generate a response. To address this issue, we provide a baseline model consisting of three modules: retrieving the top-K relevant paragraphs from the corpus based on the dialogue history, ranking the top-N most relevant paragraphs, and concatenating them with the dialogue history to generate a response using a generation module.

**This Project contains the Vietnamese data for fine-tuning the rerank module.**






### Dataset Format

Each piece of data contains three attributes: query, positive, and negative.

The query is a concatenation of the conversation history in reverse order, with the last turn marked as "<last_turn>", and the rest marked with "<user>" for user input and "<agent>" for system output. For example：
'<last_turn> Ai đã giới thiệu một trong những hệ thống phúc lợi đầu tiên cho giai cấp công nhân vào năm 1883? <agent> Đại khủng hoảng, khi các biện pháp cứu trợ khẩn cấp đã được giới thiệu dưới thời Tổng thống Franklin D. Roosevelt. <user> Khi nào Hoa Kỳ có một hệ thống phúc lợi xã hội có tổ chức?'




"Positive" refers to the positive samples, which are the samples used to label the targets of the dialogue. The titles in reverse chronological order are concatenated and separated by "//". For example:
'Otto von Bismarck, Thủ tướng Đức, giới thiệu một trong những hệ thống phúc lợi đầu tiên cho các tầng lớp lao động vào năm 1883. // Lịch sử[sửa | sửa mã nguồn] // An sinh xã hội – Wikipedia tiếng Việt // vi-SocialSecurity'


 "Negative" refers to negative samples, which are obtained by retrieving the passage with the highest BM25 value from the dialogue history, excluding the positive samples. The format is the same as for positive samples.


### 数据集加载方式
通过代码范例等方式，提供数据集通过MaaS/Dataset SDK进行加载和使用的详细说明。