--- title: Kimi-K3-mlx-reap160-2bit canonical_url: "https://www.modelscope.cn/models/mlx-community/Kimi-K3-mlx-reap160-2bit" md_url: "https://www.modelscope.cn/models/mlx-community/Kimi-K3-mlx-reap160-2bit.md" repository: mlx-community/Kimi-K3-mlx-reap160-2bit last_updated: 2026-08-03 license: other pipeline_tag: text-generation tasks: - text-generation model_type: - kimi_k3 architectures: - KimiK3ForConditionalGeneration base_model: - moonshotai/Kimi-K3 base_model_relation: finetune parameters: 50.2B tensor_type: - BF16 - U32 - F32 library_name: - mlx - safetensors - pytorch frameworks: - pytorch downloads: 17 stars: 0 tags: - mlx - kimi-k3 - moe - reap - pruned - apple-silicon - text-generation - code --- # Kimi-K3-mlx-reap160-2bit > Kimi-K3-mlx-reap160-2bit - mlx-community 在 ModelScope 开源的模型。Kimi-K3 MLX: 160 experts, 2-bit (181 GB) mlx-community/Kimi-K3-mlx-reap160-2bit 是 ModelScope 魔搭社区上的 50.2B 参数text-generation模型，采用 other 许可，基于 moonshotai/Kimi-K3 构建。 - **Repository**: mlx-community/Kimi-K3-mlx-reap160-2bit - **License**: other - **Tasks**: text-generation - **Parameters**: 50.2B - **Base model**: moonshotai/Kimi-K3 - **Tags**: mlx, kimi-k3, moe, reap, pruned, apple-silicon, text-generation, code - **Downloads**: 17 - **Stars**: 0 - **Last updated**: 2026-08-03 Source: https://www.modelscope.cn/models/mlx-community/Kimi-K3-mlx-reap160-2bit --- # Kimi-K3 MLX: 160 experts, 2-bit (181 GB) An MLX build of [`moonshotai/Kimi-K3`](https://huggingface.co/moonshotai/Kimi-K3) (2.78 T total, 104 B active) pruned to **160 of 896 routed experts per layer** and quantized to **2-bit at group 128**, so it runs **resident on a 192 GB Apple Silicon machine**. In short, about half the memory of the smallest previously published MLX tier. ## Read this before downloading **This is a code-capable model with degraded factual recall. It is not a general-purpose K3.** Measured with greedy decoding, 40 new tokens: | prompt | output | verdict | |---|---|---| | `def merge_intervals(intervals):` | `intervals = sorted(intervals, key=lambda x: x[0])` → `merged = []` → `for interval in intervals:` → `if not merged or merged[-1][1] < interval[0]:` | **correct** | | `The capital of France is` | `the capital of France is the capital of France is…` | **fails** | | `机器学习的基本原理是` | `理解数据，从数据中学习，学习是，学习，学习…` (starts coherently, then loops) | **degrades** | If you need factual recall or Chinese, use a larger tier. If you want a K3 that writes code on a 192 GB Mac, this is the only build we know of that fits. | | | |---|---| | size | **181.2 GB** (57 files) | | experts | **160 of 896** kept per layer (82% pruned), 2-bit affine, group 128 | | shared experts | both kept, never pruned | | non-expert | per-component: attention 6-bit, embeddings/router 8-bit | | load time | 53 s | | decode | 3.93 tok/s (measured on a CUDA node; Apple Silicon will differ) | | routing | top-16 of 160 = 10.0% density (base 1.8%) | ## Why the factual recall is gone, precisely A token's MoE output combines its top-k experts with gate weights **renormalized to sum to 1**, so the model cannot down-weight an expert it dislikes. With `p_j = P(expert j is in a token's top-k)`, a kept set `S` serves `Σ_{j∈S} p_j / top_k` of the router's demand. We measured this on a 32k-token calibration set: | kept | random | AIMER (weight-only) | **REAP (this build)** | ideal | |---|---|---|---|---| | 160 | 17.9% | 22.3% | **51.7%** | 54.3% | | 376 | 42.0% | 48.2% | 77.5% | 79.6% | At 160 experts a token reaches ~8.3 of the 16 experts it wanted. That is enough for code and not for facts. Recovering full capability needs ~77% retained mass ≈ **376 experts ≈ 366 GB**, which does not fit 192 GB at MLX's 2-bit floor. **So this limitation is arithmetic, not a tuning failure.** Things we tried that did **not** help, so you don't repeat them: - **Lower `top_k`** (16 → 12 → 8 → 6): the France prompt loops identically at every value, and Chinese gets *worse*. The damage is missing knowledge, not misweighted survivors. - **A weight-only criterion (AIMER)**: only 1.25× better than random. At 137 experts it retained 19.4% and every prompt looped. - **Shared-basis + low-rank factorization** (keep all 896 experts as `W_base + A_jB_j`): closed by measurement — the shared mean alone has **99.9%** relative error and rank 256 still **86%**. K3's experts do not share a basis. ## Usage ```python # pip install mlx mlx-lm # this repo bundles kimi_k3.py; register it with mlx_lm first: # cp kimi_k3.py $(python -c "import mlx_lm.models,os;print(os.path.dirname(mlx_lm.models.__file__))")/ from mlx_lm import load, generate model, tokenizer = load("./Kimi-K3-mlx-reap160-2bit") print(generate(model, tokenizer, "def merge_intervals(intervals):", max_tokens=128)) ``` Raise the Metal wired limit — it defaults to ~75% of RAM, below this build: ```bash sudo sysctl iogpu.wired_limit_mb=185000 ``` ## What has and has not been checked | check | status | |---|---| | structural: config, shard index, module coverage | done | | loads and generates | done | | code generation | done — correct | | factual recall | done — **fails** | | Chinese | done — **degrades** | | perplexity / benchmarks | **not done** | | Apple Silicon decode speed | **not done** (3.93 tok/s measured on a CUDA node) | The prune plan and the REAP saliency ship with the weights, so the selection is auditable and the retained-mass numbers above are reproducible. ## Attribution - Base model: [`moonshotai/Kimi-K3`](https://huggingface.co/moonshotai/Kimi-K3). - Loader and streaming converter vendored from [PipeNetwork/kimi-k3-mlx](https://github.com/PipeNetwork/kimi-k3-mlx). - REAP expert saliency: [arXiv 2510.13999](https://arxiv.org/abs/2510.13999). - Prior art at 512 GB: [`pipenetwork/Kimi-K3-REAP80-MLX-mxfp4-q8`](https://huggingface.co/pipenetwork/Kimi-K3-REAP80-MLX-mxfp4-q8).