---
title: EE-LLM-1B-dj-refine-300B
canonical_url: "https://www.modelscope.cn/models/Data-Juicer/EE-LLM-1B-dj-refine-300B"
md_url: "https://www.modelscope.cn/models/Data-Juicer/EE-LLM-1B-dj-refine-300B.md"
repository: Data-Juicer/EE-LLM-1B-dj-refine-300B
last_updated: 2024-02-01
license: "Apache License 2.0"
pipeline_tag: text-generation
tasks:
  - text-generation
library_name:
  - pytorch
frameworks:
  - Pytorch
downloads: 327
stars: 0
tags:
  - data-juicer
  - early-exit
  - "arxiv:2312.04916"
---

# EE-LLM-1B-dj-refine-300B

> EE-LLM-1B-dj-refine-300B - Data-Juicer 在 ModelScope 开源的模型。A 1.3B decoder-only LLM with two early exits, trained with 300B tokens from the Data-Juicer refined dataset.

Data-Juicer/EE-LLM-1B-dj-refine-300B 是 ModelScope 魔搭社区上的text-generation模型，采用 Apache License 2.0 许可。

- **Repository**: Data-Juicer/EE-LLM-1B-dj-refine-300B
- **License**: Apache License 2.0
- **Tasks**: text-generation
- **Tags**: data-juicer, early-exit, arxiv:2312.04916
- **Downloads**: 327
- **Stars**: 0
- **Last updated**: 2024-02-01

Source: https://www.modelscope.cn/models/Data-Juicer/EE-LLM-1B-dj-refine-300B

---

## Introduction

A 1.3B decoder-only LLM with two early exits, trained using [EE-LLM](https://github.com/pan-x-c/EE-LLM) with 300B tokens from the [Data-Juicer](https://github.com/alibaba/data-juicer) refined dataset.

Since ModelScope does not currently support early-exit LLM, this repository only provides the model checkpoint in EE-LLM format.
Please use EE-LLM to experiment with the inference acceleration brought by early exits.

The model is used in the experiment section of our [paper](https://arxiv.org/abs/2312.04916).
For more details, please refer to our paper.

## Usage

To use this checkpoint, please refer to [EE-LLM](https://github.com/pan-x-c/EE-LLM#inference).

## Reference

```
@misc{chen2023eellm,
    title={EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism}, 
    author={Yanxi Chen and Xuchen Pan and Yaliang Li and Bolin Ding and Jingren Zhou},
    year={2023},
    eprint={2312.04916},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

## Clone with HTTP
```bash
 git clone https://www.modelscope.cn/Data-Juicer/EE-LLM-1B-dj-refine-300B.git
```