---
title: DeepSeek-V4-Flash-GGUF
canonical_url: "https://www.modelscope.cn/models/bartowski/DeepSeek-V4-Flash-GGUF"
md_url: "https://www.modelscope.cn/models/bartowski/DeepSeek-V4-Flash-GGUF.md"
repository: bartowski/DeepSeek-V4-Flash-GGUF
last_updated: 2026-06-30
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
base_model:
  - deepseek-ai/DeepSeek-V4-Flash
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
downloads: 525
stars: 0
tags:
  - gguf
---

# DeepSeek-V4-Flash-GGUF

> DeepSeek-V4-Flash-GGUF - bartowski 在 ModelScope 开源的模型。Llamacpp Quantizations of DeepSeek-V4-Flash by deepseek-ai

bartowski/DeepSeek-V4-Flash-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 mit 许可，基于 deepseek-ai/DeepSeek-V4-Flash 构建。

- **Repository**: bartowski/DeepSeek-V4-Flash-GGUF
- **License**: mit
- **Tasks**: text-generation
- **Base model**: deepseek-ai/DeepSeek-V4-Flash
- **Tags**: gguf
- **Downloads**: 525
- **Stars**: 0
- **Last updated**: 2026-06-30

Source: https://www.modelscope.cn/models/bartowski/DeepSeek-V4-Flash-GGUF

---

## Llamacpp Quantizations of DeepSeek-V4-Flash by deepseek-ai

Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b9843">b9843</a> for quantization.

Original model: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash

This model is in MXFP4 and as such has only been provided in MXFP4 format!

No other sizes can be provided unfortunately as MXFP4 does not quantize properly.

Run in your choice of tools:

- [llama.cpp](https://github.com/ggml-org/llama.cpp)
- [ramalama](https://github.com/containers/ramalama)
- [LM Studio](https://lmstudio.ai/)
- [koboldcpp](https://github.com/LostRuins/koboldcpp)
- [Jan AI](https://www.jan.ai/)
- [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui)
- [LoLLMs](https://github.com/ParisNeo/lollms)
- [Atomic Chat](https://atomic.chat/)

Note: since it's a newly supported model, you may need to wait for an update from the developers.

## Prompt format

No prompt format found

## Download the MXFP4 files:

| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [DeepSeek-V4-Flash-MXFP4.gguf](https://huggingface.co/bartowski/DeepSeek-V4-Flash-GGUF/tree/main/DeepSeek-V4-Flash-MXFP4) | MXFP4 | 156.00GB | true | Original quality. |

## Downloading using huggingface-cli

<details>
  <summary>Click to view download instructions</summary>

First, make sure you have hugginface-cli installed:

```
pip install -U "huggingface_hub[cli]"
```

```
huggingface-cli download bartowski/DeepSeek-V4-Flash-GGUF --include "DeepSeek-V4-Flash-MXFP4*" --local-dir ./
```

You can either specify a new local-dir (DeepSeek-V4-Flash-MXFP4) or download them all in place (./)

</details>

## Credits

Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.

Thank you ZeroWw for the inspiration to experiment with embed/output.

Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski