---
title: plant-dnamamba-BPE-open_chromatin
canonical_url: "https://www.modelscope.cn/models/zhangtaolab/plant-dnamamba-BPE-open_chromatin"
md_url: "https://www.modelscope.cn/models/zhangtaolab/plant-dnamamba-BPE-open_chromatin.md"
repository: zhangtaolab/plant-dnamamba-BPE-open_chromatin
last_updated: 2024-12-15
license: CC-BY-NC-SA-4.0
pipeline_tag: text-classification
tasks:
  - text-classification
model_type:
  - Mamba
architectures:
  - MambaForSequenceClassification
parameters: 96.7M
tensor_type:
  - F32
library_name:
  - safetensors
  - pytorch
frameworks:
  - Pytorch
downloads: 861
stars: 0
tags:
  - DNA
  - biology
  - genomics
---

# plant-dnamamba-BPE-open_chromatin

> plant-dnamamba-BPE-open_chromatin - zhangtaolab 在 ModelScope 开源的模型。植物基础DNA大语言模型 (Plant foundation DNA large language models)

zhangtaolab/plant-dnamamba-BPE-open_chromatin 是 ModelScope 魔搭社区上的 96.7M 参数text-classification模型，采用 CC-BY-NC-SA-4.0 许可。

- **Repository**: zhangtaolab/plant-dnamamba-BPE-open_chromatin
- **License**: CC-BY-NC-SA-4.0
- **Tasks**: text-classification
- **Parameters**: 96.7M
- **Tags**: DNA, biology, genomics
- **Downloads**: 861
- **Stars**: 0
- **Last updated**: 2024-12-15

Source: https://www.modelscope.cn/models/zhangtaolab/plant-dnamamba-BPE-open_chromatin

---

# 植物基础DNA大语言模型 (Plant foundation DNA large language models)

The plant DNA large language models (LLMs) contain a series of foundation models based on different model architectures, which are pre-trained on various plant reference genomes.  
All the models have a comparable model size between 90 MB and 150 MB, BPE tokenizer is used for tokenization and 8000 tokens are included in the vocabulary.  


**开发者:** zhangtaolab

### Model Sources

- **Repository:** [Plant DNA LLMs](https://github.com/zhangtaolab/plant_DNA_LLMs)
- **Manuscript:** [PDLLMs: A group of tailored DNA large language models for analyzing plant genomes]() 

### Architecture

The model is trained based on the State-Space Mamba-130m model with modified tokenizer specific for DNA sequence.

This model is fine-tuned for predicting open chromatin.


### How to use

Install the runtime library first:
```bash
pip install transformers
pip install causal-conv1d<=1.2.0
pip install mamba-ssm<2.0.0
```

Since `transformers` library (version < 4.43.0) does not provide a MambaForSequenceClassification function, we wrote a script to train Mamba model for sequence classification.  
An inference code can be found in our [GitHub](https://github.com/zhangtaolab/plant_DNA_LLMs).  
Note that Plant DNAMamba model requires NVIDIA GPU to run.


### Training data
We use a custom MambaForSequenceClassification script to fine-tune the model.  
Detailed training procedure can be found in our manuscript.


#### Hardware
Model was trained on a NVIDIA RTX4090 GPU (24 GB).
