---
title: long-t5-tglobal-base
canonical_url: "https://www.modelscope.cn/models/google/long-t5-tglobal-base"
md_url: "https://www.modelscope.cn/models/google/long-t5-tglobal-base.md"
repository: google/long-t5-tglobal-base
last_updated: 2025-04-21
license: apache-2.0
pipeline_tag: text2text-generation
tasks:
  - text2text-generation
model_type:
  - longt5
architectures:
  - LongT5ForConditionalGeneration
library_name:
  - transformer
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 426
stars: 0
---

# long-t5-tglobal-base

> long-t5-tglobal-base - google 在 ModelScope 开源的模型。LongT5 (transient-global attention, base-sized model)

google/long-t5-tglobal-base 是 ModelScope 魔搭社区上的text2text-generation模型，采用 apache-2.0 许可。

- **Repository**: google/long-t5-tglobal-base
- **License**: apache-2.0
- **Tasks**: text2text-generation
- **Downloads**: 426
- **Stars**: 0
- **Last updated**: 2025-04-21

Source: https://www.modelscope.cn/models/google/long-t5-tglobal-base

---

# LongT5 (transient-global attention, base-sized model)

LongT5 model pre-trained on English language. The model was introduced in the paper [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://arxiv.org/pdf/2112.07916.pdf) by Guo et al. and first released in [the LongT5 repository](https://github.com/google-research/longt5). All the model architecture and configuration can be found in [Flaxformer repository](https://github.com/google/flaxformer) which uses another Google research project repository [T5x](https://github.com/google-research/t5x).

Disclaimer: The team releasing LongT5 did not write a model card for this model so this model card has been written by the Hugging Face team.

## Model description
LongT5 model is an encoder-decoder transformer pre-trained in a text-to-text denoising generative setting ([Pegasus-like generation pre-training](https://arxiv.org/pdf/1912.08777.pdf)). LongT5 model is an extension of [T5 model](https://arxiv.org/pdf/1910.10683.pdf), and it enables using one of the two different efficient attention mechanisms - (1) Local attention, or (2) Transient-Global attention. The usage of attention sparsity patterns allows the model to efficiently handle input sequence.

LongT5 is particularly effective when fine-tuned for text generation (summarization, question answering) which requires handling long input sequences (up to 16,384 tokens).

## Intended uses & limitations

The model is mostly meant to be fine-tuned on a supervised dataset. See the [model hub](https://huggingface.co/models?search=longt5) to look for fine-tuned versions on a task that interests you.

### How to use

```python
from transformers import AutoTokenizer, LongT5Model

tokenizer = AutoTokenizer.from_pretrained("google/long-t5-tglobal-base")
model = LongT5Model.from_pretrained("google/long-t5-tglobal-base")

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)

last_hidden_states = outputs.last_hidden_state
```

### BibTeX entry and citation info

```bibtex
@article{guo2021longt5,
  title={LongT5: Efficient Text-To-Text Transformer for Long Sequences},
  author={Guo, Mandy and Ainslie, Joshua and Uthus, David and Ontanon, Santiago and Ni, Jianmo and Sung, Yun-Hsuan and Yang, Yinfei},
  journal={arXiv preprint arXiv:2112.07916},
  year={2021}
}
```