---
title: mamba-2.8b-slimpj
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/mamba-2.8b-slimpj"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/mamba-2.8b-slimpj.md"
repository: AI-ModelScope/mamba-2.8b-slimpj
last_updated: 2024-03-07
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 4883
stars: 0
---

# mamba-2.8b-slimpj

> mamba-2.8b-slimpj - AI-ModelScope 在 ModelScope 开源的模型。Mamba-2.8b-slimpj is a model using the Mamba architecture, with 2.8B parameters, trained for 600B tokens on the SlimPajama dataset.

AI-ModelScope/mamba-2.8b-slimpj 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/mamba-2.8b-slimpj
- **License**: apache-2.0
- **Tasks**: text-generation
- **Downloads**: 4883
- **Stars**: 0
- **Last updated**: 2024-03-07

Source: https://www.modelscope.cn/models/AI-ModelScope/mamba-2.8b-slimpj

---

Mamba-2.8b-slimpj is a model using the [Mamba](https://arxiv.org/abs/2312.00752) architecture, with 2.8B parameters, trained for 600B tokens on the SlimPajama dataset. 

Model code: https://github.com/state-spaces/mamba/tree/main

To load the model, follow the installation instruction in the code repo, and then:
```
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
model = MambaLMHeadModel.from_pretrained("state-spaces/mamba-2.8b-slimpj")
```