---
title: baichuan2-13b-chat-gguf
canonical_url: "https://www.modelscope.cn/models/shaowenchen/baichuan2-13b-chat-gguf"
md_url: "https://www.modelscope.cn/models/shaowenchen/baichuan2-13b-chat-gguf.md"
repository: shaowenchen/baichuan2-13b-chat-gguf
last_updated: 2023-09-22
license: other
pipeline_tag: text2text-generation
tasks:
  - text2text-generation
model_type:
  - baichuan
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
language:
  - zh
  - en
downloads: 13537
stars: 11
tags:
  - gguf
  - baichuan
  - chinese
---

# baichuan2-13b-chat-gguf

> baichuan2-13b-chat-gguf - shaowenchen 在 ModelScope 开源的模型。There may be a problem with this model transformation, and the model's intelligence is behaving abnormally.

shaowenchen/baichuan2-13b-chat-gguf 是 ModelScope 魔搭社区上的text2text-generation模型，采用 other 许可。

- **Repository**: shaowenchen/baichuan2-13b-chat-gguf
- **License**: other
- **Tasks**: text2text-generation
- **Tags**: gguf, baichuan, chinese
- **Downloads**: 13537
- **Stars**: 11
- **Last updated**: 2023-09-22

Source: https://www.modelscope.cn/models/shaowenchen/baichuan2-13b-chat-gguf

---

There may be a problem with this model transformation, and the model's intelligence is behaving abnormally.

## Provided files

| Name                         | Quant method | Size   |
| ---------------------------- | ------------ | ------ |
| baichuan2-13b-chat.Q4_0.gguf | Q4_0         | 7.4 GB |
| baichuan2-13b-chat.Q4_1.gguf | Q4_1         | 8.2 GB |
| baichuan2-13b-chat.Q5_0.gguf | Q5_0         | 9.0 GB |
| baichuan2-13b-chat.Q5_1.gguf | Q5_1         | 9.8 GB |
| baichuan2-13b-chat.Q8_0.gguf | Q8_0         | 14 GB  |
| baichuan2-13b-chat.gguf      | full         | 26 GB  |

Usage:

```
docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/gguf-model-name.gguf hubimage/llama-cpp-python:latest
```

and you can view http://localhost:8000/docs to see the swagger UI.

## Provided images

| Name                                       | Quant method | Compressed Size |
| ------------------------------------------ | ------------ | --------------- |
| `shaowenchen/baichuan2-13b-chat-gguf:Q4_0` | Q4_0         | 7.44 GB         |
| `shaowenchen/baichuan2-13b-chat-gguf:Q4_1` | Q4_1         | 8.27 GB         |
| `shaowenchen/baichuan2-13b-chat-gguf:Q5_0` | Q5_0         | 9.29 GB         |
| `shaowenchen/baichuan2-13b-chat-gguf:Q5_1` | Q5_1         | 10 GB           |
| `shaowenchen/baichuan2-13b-chat-gguf:Q8_0` | Q8_0         | 13.6 GB         |
| `shaowenchen/baichuan2-13b-chat-gguf:full` | full         | 20.12 GB        |

Usage:

```
docker run --rm -p 8000:8000 shaowenchen/baichuan2-13b-chat-gguf:Q4_0
```

and you can view http://localhost:8000/docs to see the swagger UI.