---
title: FastMTP
canonical_url: "https://www.modelscope.cn/models/TencentBAC/FastMTP"
md_url: "https://www.modelscope.cn/models/TencentBAC/FastMTP.md"
repository: TencentBAC/FastMTP
last_updated: 2025-09-16
license: apache-2.0
model_type:
  - mimo
architectures:
  - MiMoForCausalLM
parameters: 7.8B
tensor_type:
  - BF16
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
inference_backends:
  - "deploy_task text/emb"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 165
stars: 6
---

# FastMTP

> FastMTP - TencentBAC 在 ModelScope 开源的模型。FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction

TencentBAC/FastMTP 是 ModelScope 魔搭社区上的 7.8B 参数机器学习模型，采用 apache-2.0 许可，可用 deploy_task text/emb、sglang 0.5.2、vllm 0.9.2 部署。

- **Repository**: TencentBAC/FastMTP
- **License**: apache-2.0
- **Parameters**: 7.8B
- **Inference backends**: deploy_task text/emb, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 165
- **Stars**: 6
- **Last updated**: 2025-09-16

Source: https://www.modelscope.cn/models/TencentBAC/FastMTP

---

# FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction

<p align="left">
  <strong>Technical report (coming soon)</strong> ·
  <a href="https://github.com/Tencent-BAC/FastMTP"><strong>Github</strong></a> ·
  <a href="https://huggingface.co/TencentBAC/FastMTP"><strong>HuggingFace</strong></a> ·
  <a href="https://modelscope.cn/models/TencentBAC/FastMTP"><strong>ModelScope</strong></a>
</p>

## Overview

FastMTP is a simple yet effective method that enhances Multi-Token Prediction (MTP) for speculative decoding during inference. Our approach fine-tunes a single MTP head with shared weights across multiple causal draft steps, enabling it to capture longer-range dependencies and achieve higher acceptance rates in speculative decoding. By incorporating language-aware vocabulary compression, we further reduce computational overhead during draft generation. Experimental results across diverse benchmarks demonstrate that FastMTP achieves an average of 2.03× speedup over vanilla next token prediction while maintaining lossless output quality. With low training cost and seamless integration into existing inference frameworks, FastMTP offers a practical and rapidly deployable solution for accelerating LLM inference.

<!-- ![Model](./assets/mtp-overview.png){width=50%} -->
<img src="./assets/mtp-overview.png" width="75%">

Speedup comparison of different methods across subtasks, evaluated on a single A10 GPU:

<img src="./assets/radar_chart.png" width="55%">

## What's Included

This repository contains the model checkpoints for FastMTP, and the processed compressed vocabulary.

## Links

- Technical report (coming soon)
- Training & inference code: [GitHub Repository](https://github.com/Tencent-BAC/FastMTP)