--- title: MiniCPM-o-4_5-MFQ canonical_url: "https://www.modelscope.cn/models/Tylogi/MiniCPM-o-4_5-MFQ" md_url: "https://www.modelscope.cn/models/Tylogi/MiniCPM-o-4_5-MFQ.md" repository: Tylogi/MiniCPM-o-4_5-MFQ chinese_name: MiniCPM-o-4_5-MFQ last_updated: 2026-08-18 license: apache-2.0 pipeline_tag: any-to-any tasks: - any-to-any base_model: - OpenBMB/MiniCPM-o-4_5 base_model_relation: quantized downloads: 6 stars: 0 tags: - mfq - quantization - minicpm-o - multimodal - full-duplex --- # MiniCPM-o-4_5-MFQ > MiniCPM-o-4_5-MFQ - Tylogi 在 ModelScope 开源的模型。Run MiniCPM-o 4.5 with TyloQuant MFQ Tylogi/MiniCPM-o-4_5-MFQ 是 ModelScope 魔搭社区上的any-to-any模型，采用 apache-2.0 许可，基于 OpenBMB/MiniCPM-o-4_5 构建。 - **Repository**: Tylogi/MiniCPM-o-4_5-MFQ - **License**: apache-2.0 - **Tasks**: any-to-any - **Base model**: OpenBMB/MiniCPM-o-4_5 - **Tags**: mfq, quantization, minicpm-o, multimodal, full-duplex - **Downloads**: 6 - **Stars**: 0 - **Last updated**: 2026-08-18 Source: https://www.modelscope.cn/models/Tylogi/MiniCPM-o-4_5-MFQ --- ## Run MiniCPM-o 4.5 with TyloQuant MFQ

TyloQuant MFQ provides neuron-anchored mixed-format quantization and direct CUDA/C++ inference for large models.

This repository contains MFQ-quantized weights derived from the official MiniCPM-o 4.5 release.
Each MFQ file contains the quantized language backbone plus the BF16 vision encoder, audio encoder, Resampler, audio projector and TTS audio-token generator.
The final Token2Wav waveform decoder is a shared runtime asset and is intentionally distributed separately; it is not stored in the MFQ file.
Seven size tiers are available for different memory and quality requirements.
See the project documentation for runtime, format and deployment instructions.

### File sizes and language-backbone sizes The MFQ file size is not the size of the quantized language model alone. Every file also bundles about **2.411 GB** of fixed multimodal tensors and runtime assets. The `LLM backbone` column provides the comparable language-only size. | MFQ file | Complete MFQ (GB) | LLM backbone (GB) | LLM BPW | | :--- | ---: | ---: | ---: | | `MiniCPM-o-4_5-S4.mfq` | 7.195 | 4.783 | 4.673 | | `MiniCPM-o-4_5-S4-S.mfq` | 7.222 | 4.811 | 4.700 | | `MiniCPM-o-4_5-S4-M.mfq` | 7.450 | 5.039 | 4.923 | | `MiniCPM-o-4_5-S5-S.mfq` | 8.154 | 5.743 | 5.610 | | `MiniCPM-o-4_5-S5-M.mfq` | 8.287 | 5.875 | 5.740 | | `MiniCPM-o-4_5-S6.mfq` | 9.168 | 6.757 | 6.601 | | `MiniCPM-o-4_5-S8.mfq` | 10.959 | 8.547 | 8.350 |

_{All sizes use decimal GB. BPW is calculated over the LLM weights only.}

### Apple Metal multimodal throughput The following curve compares MFQ Metal S4 with official llama.cpp-omni Q4_0 on Apple Silicon. Each point is the mean of three fresh-process Video-MME runs matched by actual LLM prompt/KV length; error bars show the observed range.

MiniCPM-o 4.5 multimodal Prefill and Decode throughput on Apple Metal

### Quantized-model quality MFQ S4 and official GGUF Q4_0 were evaluated with identical official benchmark protocols on Video-MME, Daily-Omni and Seed-TTS-Eval ZH.

MFQ S4 versus official GGUF Q4_0 benchmark quality

### Language KLD against official llama.cpp GGUF Lower KLD is better. The curve includes all seven MFQ tiers and the complete official GGUF comparison set; legacy GGUF formats and K-quants are drawn as separate series. The table below highlights the strongest corresponding official GGUF variant for each MFQ tier under the same evaluation setup.

MiniCPM-o 4.5 language-backbone KLD by bits per weight

| MFQ tier | MFQ LLM BPW | MFQ KLD | Official GGUF | GGUF LLM BPW | GGUF KLD | KLD reduction | | :--- | ---: | ---: | :--- | ---: | ---: | ---: | | S4 | 4.673 | **0.127388** | Q4_1 | 5.125 | 0.270871 | 53.0% | | S4-S | 4.700 | **0.112949** | Q4_K_S | 4.690 | 0.209602 | 46.1% | | S4-M | 4.923 | **0.0945408** | Q4_K_M | 4.911 | 0.165195 | 42.8% | | S5-S | 5.610 | **0.0429509** | Q5_0 | 5.587 | 0.095004 | 54.8% | | S5-M | 5.740 | **0.0368912** | Q5_K_M | 5.715 | 0.139538 | 73.6% | | S6 | 6.601 | **0.0228194** | Q6_K | 6.569 | 0.038334 | 40.5% | | S8 | 8.350 | **0.0181372** | Q8_0 | 8.507 | 0.031338 | 42.1% |

_{Evaluation: WikiText-2 raw test, no BOS insertion, context and batch size 512, 583 independent chunks, 255 scored tokens per chunk and 148,665 scored tokens in total. All candidates are measured against the same official F16 GGUF reference logits using forward KLD. BPW covers the 8.189B-parameter Qwen3 language backbone only; the bundled 16-bit multimodal components are excluded.}

### What is inside each MFQ file The following components are identical in all seven tiers. Their weights are retained in BF16; only the Qwen3-8B language backbone changes between tiers. | Bundled component | Precision | Size (GB) | | :--- | :---: | ---: | | VPM / SigLIP2 vision encoder | BF16 | 0.836 | | APM / Whisper audio encoder | BF16 | 0.614 | | Vision Resampler | BF16 | 0.178 | | Audio projection layer | BF16 | 0.042 | | TTS audio-token generator | BF16 | 0.695 | | **Auxiliary model tensors subtotal** | **BF16** | **2.365** | | Embedded tokenizer, configuration and position table | Mixed/blob | 0.046 | | **All bundled non-LLM records** | | **2.411** | In short: `complete MFQ size = quantized LLM backbone + 2.411 GB bundled non-LLM records`, plus negligible container metadata. The language embedding and output head are part of the LLM-backbone figures. For example, S4 stores the language embedding as NINT4 and the output head as NINT6, while S8 stores both as NINT8. The separate TTS embedding remains BF16. ### What is not inside the MFQ file The full **Token2Wav waveform decoder is not embedded**. The bundled `tts.*` module generates audio tokens; the external Token2Wav model converts those tokens into 24 kHz waveform audio. Keeping this decoder as a shared runtime asset avoids duplicating it in every compatible MFQ model. The external asset footprint depends on the selected runtime packaging: - The PyTorch/ONNX StepAudio asset bundle used by the MFQ runtime is approximately **1.231 GB**. - The alternative OpenBMB llama.cpp Token2Wav GGUF bundle is approximately **0.918 GB**. Its current tensors are stored as F32; the GGUF conversion is containerization rather than low-bit quantization. These external files are required for waveform synthesis but are not included in, and must not be added to, the MFQ sizes in the table above. ### Language-backbone size reference against llama.cpp This comparison excludes vision, audio, TTS and Token2Wav from both sides. Hugging Face displays the GGUF sizes rounded. | Tier | MFQ language backbone | Comparable llama.cpp GGUF | | :--- | ---: | ---: | | S4 vs Q4_K_M | **4.783 GB** | 5.03 GB | | S5-M vs Q5_K_M | **5.875 GB** | 5.85 GB | | S8 vs Q8_0 | **8.547 GB** | 8.71 GB | Reference: [OpenBMB MiniCPM-o 4.5 GGUF repository](https://huggingface.co/openbmb/MiniCPM-o-4_5-gguf/tree/main). --- # MiniCPM-o 4.5 A Gemini 2.5 Flash level MLLM for vision, speech and full-duplex multimodal live streaming on your phone. [GitHub](https://github.com/OpenBMB/MiniCPM-o) | [CookBook](https://github.com/OpenSQZ/MiniCPM-V-CookBook) | [Omni-modal Demo](https://minicpmo45.modelbest.cn/) | [Audio CaseBook](https://openbmb.github.io/minicpm-o-4_5/) | [Omni Full-Duplex CaseBook](https://openbmb.github.io/minicpm-o-4_5-omni/) ## Introduction **MiniCPM-o 4.5** is the latest and most capable model in the MiniCPM-o series. It is built end to end from SigLIP2, Whisper-medium, CosyVoice2 and Qwen3-8B, with 9B parameters in total. It introduces full-duplex multimodal live streaming alongside improved visual and speech capabilities. - **Vision:** high-resolution image, video, OCR, instruct and thinking capabilities. - **Speech:** bilingual real-time conversation, configurable voices, voice cloning and role play. - **Full duplex:** continuous video and audio input while generating concurrent text and speech output. - **Proactive interaction:** continuous scene understanding with model-initiated reminders and comments. ## Model Architecture MiniCPM-o 4.5 connects its modality encoders, modality decoders and LLM through hidden states. Its full-duplex path synchronizes continuous input and output streams on a millisecond timeline and models them through time-division multiplexing in the LLM backbone.

## Evaluation

_{These figures report evaluation results for the original MiniCPM-o 4.5 model. They are not MFQ quantization-quality measurements.}

## License The MiniCPM-o/V model weights and code are licensed under the [Apache License 2.0](LICENSE). ## Citation ```bibtex @article{yao2024minicpm, title={MiniCPM-V: A GPT-4V Level MLLM on Your Phone}, author={Yao, Yuan and Yu, Tianyu and Zhang, Ao and Wang, Chongyi and Cui, Junbo and Zhu, Hongji and Cai, Tianchi and Li, Haoyu and Zhao, Weilin and He, Zhihui and others}, journal={arXiv preprint arXiv:2408.01800}, year={2024} } ```