---
title: dnabert2-conservation
canonical_url: "https://www.modelscope.cn/models/zhangtaolab/dnabert2-conservation"
md_url: "https://www.modelscope.cn/models/zhangtaolab/dnabert2-conservation.md"
repository: zhangtaolab/dnabert2-conservation
last_updated: 2024-07-10
license: CC-BY-NC-SA-4.0
pipeline_tag: text-classification
tasks:
  - text-classification
model_type:
  - BERT
architectures:
  - BertForSequenceClassification
parameters: 117.1M
tensor_type:
  - F32
library_name:
  - transformer
  - safetensors
  - pytorch
frameworks:
  - Pytorch
inference_backends:
  - "deploy_task emb"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 347
stars: 0
tags:
  - DNA
  - biology
  - genomics
---

# dnabert2-conservation

> dnabert2-conservation - zhangtaolab 在 ModelScope 开源的模型。植物基础DNA大语言模型 (Plant foundation DNA large language models)

zhangtaolab/dnabert2-conservation 是 ModelScope 魔搭社区上的 117.1M 参数text-classification模型，采用 CC-BY-NC-SA-4.0 许可，可用 deploy_task emb、sglang 0.5.2、vllm 0.9.2 部署。

- **Repository**: zhangtaolab/dnabert2-conservation
- **License**: CC-BY-NC-SA-4.0
- **Tasks**: text-classification
- **Parameters**: 117.1M
- **Inference backends**: deploy_task emb, sglang 0.5.2, vllm 0.9.2
- **Tags**: DNA, biology, genomics
- **Downloads**: 347
- **Stars**: 0
- **Last updated**: 2024-07-10

Source: https://www.modelscope.cn/models/zhangtaolab/dnabert2-conservation

---

# 植物基础DNA大语言模型 (Plant foundation DNA large language models)

The plant DNA large language models (LLMs) contain a series of foundation models based on different model architectures, which are pre-trained on various plant reference genomes.  
All the models have a comparable model size between 90 MB and 150 MB, BPE tokenizer is used for tokenization and 8000 tokens are included in the vocabulary.  


**开发者:** zhangtaolab

### Model Sources

- **Repository:** [Plant DNA LLMs](https://github.com/zhangtaolab/plant_DNA_LLMs)
- **Manuscript:** [Versatile applications of foundation DNA large language models in plant genomes]() 

### Architecture

The model is trained based on the zhihan1996/DNABERT-2-117M model with modified tokenizer.

This model is fine-tuned for predicting sequence conservation.

### How to use

Install the runtime library first:
```bash
pip install transformers
```

Here is a simple code for inference:
```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline

model_name = 'dnabert2-conservation'
# load model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained(f'zhangtaolab/{model_name}', trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(f'zhangtaolab/{model_name}', trust_remote_code=True)

# inference
sequences = ['ACATGCTAAATTAGTTGGCAATTTTTTCTCAGGTAGCTGGGCACAATTTGGTAGTCCAGTTGAACAAAATCCATTAGCTTCTTTTAGCAAGTCCCCTGGTTTGGGCCCTGCCAGTCCCATTAATACCAACCATTTGTCTGGATTGGCTGCAATTCTTTCCCCACAAGCAACAACCTCTACCAAGATTGCACCGATTGGCAAGGACCCTGGAAGGGCTGCAAATCAGATGTTTTCTAACTCTGGATCAACACAAGGAGCAGCTTTTCAGCATTCTATATCCTTTCCTGAGCAAAATGTAAAGGCAAGTCCTAGGCCTATATCTACTTTTGGTGAATCAAGTTCTAGTGCATCAAGTATTGGAACACTGTCCGGTCCTCAATTTCTTTGGGGAAGCCCAACTCCTTACTCTGAGCATTCAAACACTTCTGCCTGGTCTTCATCTTCGGTGGGGCTTCCATTTACATCTAGTGTCCAAAGGCAGGGTTTCCCATATACTAGTAATCACAGTCCTTTTCTTGGCTCCCACTCTCATCATCATGTTGGATCTGCTCCATCTGGCCTTCCGCTTGATAGGCATTTTAGCTACTTCCCTGAGTCACCTGAAGCTTCTCTCATGAGCCCGGTTGCATTTGGGAATTTAAATCACGGTGATGGGAATTTTATGATGAACAACATTAGTGCTCGTGCATCTGTAGGAGCCGGTGTTGGTCTTTCTGGAAATACCCCTGAAATTAGTTCACCCAATTTCAGAATGATGTCTCTGCCTAGGCATGGTTCCTTGTTCCATGGAAATAGTTTGTATTCTGGACCTGGAGCAACTAACATTGAGGGATTAGCTGAACGTGGACGAAGTAGACGACCTGAAAATGGTGGGAACCAAATTGATAGTAAGAAGCTGTACCAGCTTGATCTTGACAAAATCGTCTGTGGTGAAGATACAAGGACTACTTTAATGATTAAAAACATTCCTAACAAGTAAGAATAACTAAACATCTATCCT',
             'GTCGCAAAAATTGGGCCACTTGCAGTTCAATCTGTTTAATCAAAATTGCATGTGTATCAACTTTTTGCCCAATACTAGCTATATCACACCTCAACTCTTTAATGTGTTCATCACTAGTGTCGAACCTCCTCATCATTTTGTCCAACATATCCTCAACTCGCGCCATACTATCTCCACCATCCCTAGGAGTAACTTCACGATTTTGAGGAGGGACATAGGGCCCATTCCTGTCGTTTCTATTAGCATAGTTACTCCTGTTAAAGTTGTTGTCGCGGTTGTAGTTTCCATCACGTACATAATGACTCTCACGGTTGTAGTTACCATAGTTCCGACCTGGGTTCCCTTGAACTTGGCGCCAGTTATCCTGATTTGAGCCTTGGGCGCTTGGTCGGAAACCCCCTGTCTGCTCATTTACTGCATAAGTGTCCTCCGCGTAACATCATTAGGAGGTGGTGGTTTAGCAAAGTAGTTGACTGCATTTATCTTTTCTGCACCCCCTGTGACATTTTTTAGTACCAACCCAAGCTCAGTTCTCATCTGAGACATTTCTTCTCGAATCTCATCTGTGGCTCGGTTGTGAGTGGACTGCACTACGAAGGTGTTTTTCCCTGTATCAAACTTCCTAGTACTCCAAGCTTTGTTATTTCGGGAGATTTTCTCTAGTTTTTCTGCAATCTCAACATAAGTGCATTCTCCATAAGATCCACCTGCTATAGTGTCCAACACCGCTTTATTGTTATCATCCTGTCCCCGATAGAAGTATTCCTTCAGTGACTCATCATCTATACGGTGATTTAGAACACTTCTCAAGAATGAGGTGAATCTATCCCAAGAACTACTAACTAACTCTCCTGGTAGTGCCACAAAGCTGTTCACCCTTTCTTTGTGGTTTAACTTCTTGGAGATCGGATAGTAGCGTGCTAAGAAGACATCCCTTAGTTGGTTCCAAGTGAATATGGAGTTGTATGCGAGCTTAGTGAACCACATTGCAGCCTCTCCC']
pipe = pipeline('text-classification', model=model, tokenizer=tokenizer,
                trust_remote_code=True, top_k=None)
results = pipe(sequences)
print(results)

```


### Training data
We use BertForSequenceClassification to fine-tune the model.  
Detailed training procedure can be found in our manuscript.


#### Hardware
Model was trained on a NVIDIA GTX1080Ti GPU (11 GB).