---
title: Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct.md"
repository: AI-ModelScope/Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct
last_updated: 2025-09-04
license: llama3.1
model_type:
  - mlp_speculator
architectures:
  - ArcticLSTMSpeculatorPreTrainedModel
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 486
stars: 0
---

# Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct

> Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct - AI-ModelScope 在 ModelScope 开源的模型。Build a fastest OSS vllm-based speculative decoding system for your own model, using ArcticTraining and ArcticInference!

AI-ModelScope/Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct 是 ModelScope 魔搭社区上的机器学习模型，采用 llama3.1 许可。

- **Repository**: AI-ModelScope/Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct
- **License**: llama3.1
- **Downloads**: 486
- **Stars**: 0
- **Last updated**: 2025-09-04

Source: https://www.modelscope.cn/models/AI-ModelScope/Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct

---

# ArcticSpeculator

Build a fastest OSS vllm-based speculative decoding system for your own model, using [ArcticTraining](https://github.com/snowflakedb/ArcticTraining) and [ArcticInference](https://github.com/snowflakedb/ArcticInference)!

We compare the throughput (tokens/s) of existing vllm-based speculative decoding systems for Llama3.1-70B-Instruct on 8xH100 as below:

| method                                 | ShareGPT      | HumanEval |
|--------------------------------------|----------------|--------------|
| VLLM V1 Baseline      | 84.1 | 84.1    |
| VLLM V1 Eagle | 102.2   | 112.0    |
| VLLM V1 Eagle3  | 77.7   | 85.3 |
| VLLM V0 MLP-Speculator (IBM) | 77.9   | 66.7        |
| ArcticSpeculator                          | **172.4**   | **203.7**    |

For more details about ArcticSpeculator and how to use it:

* ❄️ [Using Arctic-Inference and Arctic-Training for improving real-world speculative decoding Performance (blog)]()
* 🚀 [Getting started guide using ArcticTraining](https://github.com/snowflakedb/ArcticTraining/tree/mlp-variant-speculator/projects/mlp_variant_speculator)

We also release ArcticSpeculator checkpoints we trained with [ArcticTraining](https://github.com/snowflakedb/ArcticTraining) to run with [ArcticInference](https://github.com/snowflakedb/ArcticInference):

| model | ArcticSpeculator |
|---- | ---- |
| [Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) | [Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct](https://huggingface.co/Snowflake/Arctic-LSTM-Speculator-Llama-3.1-70B-Instruct) |
| [Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | [Arctic-LSTM-Speculator-Llama-3.3-70B-Instruct](https://huggingface.co/Snowflake/Arctic-LSTM-Speculator-Llama-3.3-70B-Instruct) |
| [Qwen2.5-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct) | [Arctic-LSTM-Speculator-Qwen2.5-32B-Instruct](https://huggingface.co/Snowflake/Arctic-LSTM-Speculator-Qwen2.5-32B-Instruct) |
| [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) | [Arctic-LSTM-Speculator-Llama-3.1-8B-Instruct](https://huggingface.co/Snowflake/Arctic-LSTM-Speculator-Llama-3.1-8B-Instruct)|