--- title: FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 canonical_url: "https://www.modelscope.cn/models/mlx-community/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8" md_url: "https://www.modelscope.cn/models/mlx-community/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8.md" repository: mlx-community/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 last_updated: 2025-02-22 license: apache-2.0 model_type: - qwen2 architectures: - Qwen2ForCausalLM base_model: - FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview base_model_relation: quantized parameters: 9.2B tensor_type: - BF16 - U32 library_name: - mlx - safetensors - pytorch frameworks: - pytorch inference_backends: - "deploy_task text/emb" - "lmdeploy 0.9.1" - "lmdeploy_turbomind 0.9.1" - "sglang 0.5.2" - "vllm 0.9.2" downloads: 676 stars: 1 tags: - mlx - code --- # FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 > FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 - mlx-community 在 ModelScope 开源的模型。27 TPS with Draft model: DeepScaleR-1.5B-Preview-Q8 mlx-community/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 是 ModelScope 魔搭社区上的 9.2B 参数机器学习模型，采用 apache-2.0 许可，基于 FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview 构建，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。 - **Repository**: mlx-community/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 - **License**: apache-2.0 - **Parameters**: 9.2B - **Base model**: FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview - **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2 - **Tags**: mlx, code - **Downloads**: 676 - **Stars**: 1 - **Last updated**: 2025-02-22 Source: https://www.modelscope.cn/models/mlx-community/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 --- 13 TPS 27 TPS with Draft model: [DeepScaleR-1.5B-Preview-Q8](https://huggingface.co/mlx-community/DeepScaleR-1.5B-Preview-Q8) oh yeah! 100% faster for math/code stuff. . Macbook M4 Max: high power (10 TPS on low-power, GPU draws only 5 watts...less than your brain) system prompt: "You are Fuse01. You answer very direct brief and concise" prompt: "Write a quick sort in C++" Context: 131072, Temp: 0 . Try this model in Visual Studio Code with the Roo Code extension. Starting in Architect Mode and letting it auto switch to Code Mode.... it actually spits decent code for small projects with multiple files. Getting close to last year's Claude Sonnet for small projects. It actually stays reasonably stable even with Roo Code's huge 10k system prompt. The model still shits the bed for big projects but better after adding roo-code-memory-bank. So far (Feb 20, 2025) this is the only model & quant that runs fast on Mac, spits decent code on projects AND works with Speculative Decoding. Huge thanks to all who helped Macs get this far! # bobig/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8 The Model [bobig/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8](https://huggingface.co/bobig/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8) was converted to MLX format from [FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview) using mlx-lm version **0.21.4**. (FYI: the mlx-lm version should be the same in Base model and Draft model) ## Use with mlx ```bash pip install mlx-lm ``` ```python from mlx_lm import load, generate model, tokenizer = load("bobig/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q8") prompt = "hello" if tokenizer.chat_template is not None: messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) response = generate(model, tokenizer, prompt=prompt, verbose=True) ``` Are you still reading down here? Maybe check out this new Q4 lossless from NexaAI and tell the MLX community how to improve mlx-lm to get 8-bit quality at 4-bit speed! [DeepSeek-R1-Distill-Qwen-1.5B-NexaQuant](https://huggingface.co/NexaAIDev/DeepSeek-R1-Distill-Qwen-1.5B-NexaQuant) ore