---
title: mbart-large-50
canonical_url: "https://www.modelscope.cn/models/facebook/mbart-large-50"
md_url: "https://www.modelscope.cn/models/facebook/mbart-large-50.md"
repository: facebook/mbart-large-50
last_updated: 2025-05-20
license: mit
pipeline_tag: text2text-generation
tasks:
  - text2text-generation
model_type:
  - mbart
architectures:
  - MBartForConditionalGeneration
library_name:
  - transformer
  - tensorflow
  - pytorch
frameworks:
  - pytorch
language:
  - multilingual
  - ar
  - cs
  - de
  - en
  - es
  - et
  - fi
  - fr
  - gu
  - hi
  - it
  - ja
  - kk
  - ko
  - lt
  - lv
  - my
  - ne
  - nl
  - ro
  - ru
  - si
  - tr
  - vi
  - zh
  - af
  - az
  - bn
  - fa
  - he
  - hr
  - id
  - ka
  - km
  - mk
  - ml
  - mn
  - mr
  - pl
  - ps
  - pt
  - sv
  - sw
  - ta
  - te
  - th
  - tl
  - uk
  - ur
  - xh
  - gl
  - sl
downloads: 597
stars: 0
tags:
  - mbart-50
---

# mbart-large-50

> mbart-large-50 - facebook 在 ModelScope 开源的模型。mBART-50 is a multilingual Sequence-to-Sequence model pre-trained using the "Multilingual Denoising Pretraining" objective. It was introduced in Multilingual Translation with Extensible Multilingual Pretraining…

facebook/mbart-large-50 是 ModelScope 魔搭社区上的text2text-generation模型，采用 mit 许可。

- **Repository**: facebook/mbart-large-50
- **License**: mit
- **Tasks**: text2text-generation
- **Tags**: mbart-50
- **Downloads**: 597
- **Stars**: 0
- **Last updated**: 2025-05-20

Source: https://www.modelscope.cn/models/facebook/mbart-large-50

---

# mBART-50

mBART-50 is a multilingual Sequence-to-Sequence model pre-trained using the "Multilingual Denoising Pretraining" objective. It was introduced in [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://arxiv.org/abs/2008.00401) paper.

## Model description

mBART-50 is a multilingual Sequence-to-Sequence model. It was introduced to show that multilingual translation models can be created through multilingual fine-tuning. 
Instead of fine-tuning on one direction, a pre-trained model is fine-tuned on many directions simultaneously. mBART-50 is created using the original mBART model and extended to add extra 25 languages to support multilingual machine translation models of 50 languages. The pre-training objective is explained below.

**Multilingual Denoising Pretraining**: The model incorporates N languages by concatenating data: 
`D = {D1, ..., DN }` where each Di is a collection of monolingual documents in language `i`. The source documents are noised using two schemes, 
first randomly shuffling the original sentences' order, and second a novel in-filling scheme, 
where spans of text are replaced with a single mask token. The model is then tasked to reconstruct the original text. 
35% of each instance's words are masked by random sampling a span length according to a Poisson distribution `(λ = 3.5)`.
The decoder input is the original text with one position offset. A language id symbol `LID` is used as the initial token to predict the sentence.


## Intended uses & limitations

`mbart-large-50` is pre-trained model and primarily aimed at being fine-tuned on translation tasks. It can also be fine-tuned on other multilingual sequence-to-sequence tasks. 
See the [model hub](https://huggingface.co/models?filter=mbart-50) to look for fine-tuned versions.


## Training

As the model is multilingual, it expects the sequences in a different format. A special language id token is used as a prefix in both the source and target text. The text format is `[lang_code] X [eos]` with `X` being the source or target text respectively and `lang_code` is `source_lang_code` for source text and `tgt_lang_code` for target text. `bos` is never used. Once the examples are prepared in this format, it can be trained as any other sequence-to-sequence model.


```python
from transformers import MBartForConditionalGeneration, MBart50TokenizerFast

model = MBartForConditionalGeneration.from_pretrained("facebook/mbart-large-50")
tokenizer = MBart50TokenizerFast.from_pretrained("facebook/mbart-large-50", src_lang="en_XX", tgt_lang="ro_RO")

src_text = " UN Chief Says There Is No Military Solution in Syria"
tgt_text =  "Şeful ONU declară că nu există o soluţie militară în Siria"

model_inputs = tokenizer(src_text, return_tensors="pt")
with tokenizer.as_target_tokenizer():
    labels = tokenizer(tgt_text, return_tensors="pt").input_ids

model(**model_inputs, labels=labels) # forward pass
```



## Languages covered
Arabic (ar_AR), Czech (cs_CZ), German (de_DE), English (en_XX), Spanish (es_XX), Estonian (et_EE), Finnish (fi_FI), French (fr_XX), Gujarati (gu_IN), Hindi (hi_IN), Italian (it_IT), Japanese (ja_XX), Kazakh (kk_KZ), Korean (ko_KR), Lithuanian (lt_LT), Latvian (lv_LV), Burmese (my_MM), Nepali (ne_NP), Dutch (nl_XX), Romanian (ro_RO), Russian (ru_RU), Sinhala (si_LK), Turkish (tr_TR), Vietnamese (vi_VN), Chinese (zh_CN), Afrikaans (af_ZA), Azerbaijani (az_AZ), Bengali (bn_IN), Persian (fa_IR), Hebrew (he_IL), Croatian (hr_HR), Indonesian (id_ID), Georgian (ka_GE), Khmer (km_KH), Macedonian (mk_MK), Malayalam (ml_IN), Mongolian (mn_MN), Marathi (mr_IN), Polish (pl_PL), Pashto (ps_AF), Portuguese (pt_XX), Swedish (sv_SE), Swahili (sw_KE), Tamil (ta_IN), Telugu (te_IN), Thai (th_TH), Tagalog (tl_XX), Ukrainian (uk_UA), Urdu (ur_PK), Xhosa (xh_ZA), Galician (gl_ES), Slovene (sl_SI)


## BibTeX entry and citation info
```
@article{tang2020multilingual,
    title={Multilingual Translation with Extensible Multilingual Pretraining and Finetuning},
    author={Yuqing Tang and Chau Tran and Xian Li and Peng-Jen Chen and Naman Goyal and Vishrav Chaudhary and Jiatao Gu and Angela Fan},
    year={2020},
    eprint={2008.00401},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```