---
title: Ring-flash-linear-2.0-GPTQ-int4
canonical_url: "https://www.modelscope.cn/models/inclusionAI/Ring-flash-linear-2.0-GPTQ-int4"
md_url: "https://www.modelscope.cn/models/inclusionAI/Ring-flash-linear-2.0-GPTQ-int4.md"
repository: inclusionAI/Ring-flash-linear-2.0-GPTQ-int4
last_updated: 2025-12-30
license: mit
pipeline_tag: text-generation
tasks:
- text-generation
model_type:
- bailing_moe_linear
architectures:
- BailingMoeLinearV2ForCausalLM
base_model:
- inclusionAI/Ring-mini-linear-2.0
base_model_relation: quantized
parameters: 15.4B
tensor_type:
- I64
- I32
- BF16
library_name:
- transformer
- safetensors
- pytorch
frameworks:
- Pytorch
language:
- en
downloads: 119
stars: 3
---
# Ring-flash-linear-2.0-GPTQ-int4
> Ring-flash-linear-2.0-GPTQ-int4 - inclusionAI 在 ModelScope 开源的模型。Quantized Ring-Linear-2.0
inclusionAI/Ring-flash-linear-2.0-GPTQ-int4 是 ModelScope 魔搭社区上的 15.4B 参数text-generation模型，采用 mit 许可，基于 inclusionAI/Ring-mini-linear-2.0 构建。
- **Repository**: inclusionAI/Ring-flash-linear-2.0-GPTQ-int4
- **License**: mit
- **Tasks**: text-generation
- **Parameters**: 15.4B
- **Base model**: inclusionAI/Ring-mini-linear-2.0
- **Downloads**: 119
- **Stars**: 3
- **Last updated**: 2025-12-30
Source: https://www.modelscope.cn/models/inclusionAI/Ring-flash-linear-2.0-GPTQ-int4
---
# Quantized Ring-Linear-2.0
## Introduction
To enable deployment of [Ring-Linear-2.0](https://github.com/inclusionAI/Ring-V2/blob/main/hybrid_linear/README.md
) on memory-constrained devices, we release quantized weights using the GPTQ INT4 format. Additionally, we evaluate the online FP8 quantization performance of `Ring-Linear-2.0` models, which closely approaches that of BF16 precision.
## Model Downloads
| **Model** | **Maximum Supported Length** | **Download** |
|:----------------------:| :----------------: |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| Ring-flash-linear-2.0-GPTQ-int4 | 128k | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ring-flash-linear-2.0-GPTQ-int4)
[🤖 ModelScope](https://www.modelscope.cn/models/inclusionAI/Ring-flash-linear-2.0-GPTQ-int4) |
| Ring-mini-linear-2.0-GPTQ-int4 | 512k | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ring-mini-linear-2.0-GPTQ-int4)
[🤖 ModelScope](https://www.modelscope.cn/models/inclusionAI/Ring-mini-linear-2.0-GPTQ-int4) |
## Quickstart
### 🚀 vLLM
#### Environment Preparation
Since the Pull Request (PR) has not been submitted to the vLLM community at this stage, please prepare the environment by following the steps below.
First, create a Conda environment with Python 3.10 and CUDA 12.8:
```shell
conda create -n vllm python=3.10
conda activate vllm
```
Next, install our vLLM wheel package:
```shell
pip install https://media.githubusercontent.com/media/zheyishine/vllm_whl/refs/heads/main/vllm-0.8.5.post2.dev28%2Bgd327eed71.cu128-cp310-cp310-linux_x86_64.whl --force-reinstall
```
Finally, install compatible versions of transformers after vLLM is installed:
```shell
pip install transformers==4.51.1
```
#### Offline Inference
```python
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
if __name__ == '__main__':
tokenizer = AutoTokenizer.from_pretrained("inclusionAI/Ring-flash-linear-2.0-GPTQ-int4")
sampling_params = SamplingParams(temperature=0.6, top_p=1.0, max_tokens=16384)
# use `max_num_seqs=1` without concurrency
llm = LLM(model="inclusionAI/Ring-flash-linear-2.0-GPTQ-int4", dtype='auto', enable_prefix_caching=False, max_num_seqs=128)
prompt = "Give me a short introduction to large language models."
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
outputs = llm.generate([text], sampling_params)
for output in outputs:
print(output.outputs[0].text)
```
#### Online Inference
```shell
vllm serve inclusionAI/Ring-flash-linear-2.0-GPTQ-int4 \
--tensor-parallel-size 2 \
--pipeline-parallel-size 1 \
--gpu-memory-utilization 0.90 \
--max-num-seqs 128 \
--no-enable-prefix-caching
--api-key your-api-key
```
## Evaluation
We evaluate the INT4 and FP8 quantized models using several datasets. The FP8 quantization is applied via the quantization="fp8" argument in SGLang or vLLM.
### Ring-mini-linear-2.0
| **Dataset** | **BF16** | **FP8** | **GPTQ-Int4** |
| :----------------: |:--------:|:-------:|:-------------:|
| AIME25 | 73.65 | 72.40 | 66.56 |
| AIME24 | 79.95 | 79.53 | 74.95 |
| LiveCodeBench| 59.53 | 58.42 | 56.29 |
| GPQA | 65.69 | 66.79 | 62.53 |
### Ring-flash-linear-2.0
| **Dataset** | **BF16** | **FP8** | **GPTQ-Int4** |
| :----------------: |:--------:|:-------:| :-----------------------:|
| AIME25 | 85.10 | 84.22 | 82.88 |
| LiveCodeBench| 69.82 | 69.44 | 66.14 |
| GPQA | 72.85 | 72.95 | 71.72 |
## License
This code repository is licensed under [the MIT License](https://github.com/inclusionAI/Ring-V2/blob/master/LICENSE).
## Citation
```shell
@misc{lingteam2025attentionmattersefficienthybrid,
title={Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning},
author={Ling Team and Bin Han and Caizhi Tang and Chen Liang and Donghao Zhang and Fan Yuan and Feng Zhu and Jie Gao and Jingyu Hu and Longfei Li and Meng Li and Mingyang Zhang and Peijie Jiang and Peng Jiao and Qian Zhao and Qingyuan Yang and Wenbo Shen and Xinxing Yang and Yalin Zhang and Yankun Ren and Yao Zhao and Yibo Cao and Yixuan Sun and Yue Zhang and Yuchen Fang and Zibin Lin and Zixuan Cheng and Jun Zhou},
year={2025},
eprint={2510.19338},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2510.19338},
}
```