---
title: LongCat-Flash-Lite-Sparse-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/LongCat-Flash-Lite-Sparse-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/LongCat-Flash-Lite-Sparse-4bit.md"
repository: mlx-community/LongCat-Flash-Lite-Sparse-4bit
last_updated: 2026-08-31
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - longcat_flash_sparse
architectures:
  - LongcatCausalLM
base_model:
  - meituan-longcat/LongCat-Flash-Lite-Sparse
base_model_relation: quantized
parameters: 10.7B
tensor_type:
  - BF16
  - U32
  - F32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 28
stars: 0
tags:
  - mlx
  - longcat
  - lsa
  - moe
  - ngram-embedding
---

# LongCat-Flash-Lite-Sparse-4bit

> LongCat-Flash-Lite-Sparse-4bit - mlx-community 在 ModelScope 开源的模型。LongCat-Flash-Lite-Sparse-4bit (MLX)

mlx-community/LongCat-Flash-Lite-Sparse-4bit 是 ModelScope 魔搭社区上的 10.7B 参数text-generation模型，采用 mit 许可，基于 meituan-longcat/LongCat-Flash-Lite-Sparse 构建。

- **Repository**: mlx-community/LongCat-Flash-Lite-Sparse-4bit
- **License**: mit
- **Tasks**: text-generation
- **Parameters**: 10.7B
- **Base model**: meituan-longcat/LongCat-Flash-Lite-Sparse
- **Tags**: mlx, longcat, lsa, moe, ngram-embedding
- **Downloads**: 28
- **Stars**: 0
- **Last updated**: 2026-08-31

Source: https://www.modelscope.cn/models/mlx-community/LongCat-Flash-Lite-Sparse-4bit

---

# LongCat-Flash-Lite-Sparse-4bit (MLX)

4-bit MLX quantization of [meituan-longcat/LongCat-Flash-Lite-Sparse](https://huggingface.co/meituan-longcat/LongCat-Flash-Lite-Sparse) (69B-A3B, `LongcatCausalLM`).

4-bit (~36 GB of weights) is the smallest and fastest variant, for a 64 GB Mac. Also available: [6-bit](https://huggingface.co/AlazarM/LongCat-Flash-Lite-Sparse-6bit) (~52 GB, 96 GB Macs) and [8-bit](https://huggingface.co/AlazarM/LongCat-Flash-Lite-Sparse-8bit) (~68 GB, 128 GB Macs, near-lossless).

## What's in this checkpoint
LongCat-Flash-Lite-Sparse adds three things vanilla LongCat-Flash lacks:
- **LongCat Sparse Attention (LSA)** — a DeepSeek-style lightning indexer over MLA, with streaming-aware indexing (fixed sink + local window) and cross-layer index reuse (`cli_factor`). Native long context.
- **Zero-computation (identity) experts** in the ScMoE decoder (256 routed + 128 identity, top-12).
- **N-gram ("oe") input embedding** — ~46% of the parameters, fused into the token embedding.

## The n-gram fix
The `oe` embedding hash and tables are identical to the published n-gram references (the *Scaling Embeddings* paper, mlx-lm, SGLang, llama.cpp, Meituan's dense modeling). The one difference in `LongcatCausalLM` is the **fusion**: it keeps the word embedding at **full scale** —
`word + Σ projections / (1 + num_embedders)` — rather than the dense form `(word + Σ projections) / (1 + num_embedders)`. Dividing the word by 13 garbles generation; this build applies the correct fusion.

## Usage
Requires mlx-vlm with `longcat_flash_sparse` support ([PR #2063](https://github.com/Blaizzy/mlx-vlm/pull/2063)):

```bash
pip install git+https://github.com/Lazarus-931/mlx-vlm@add-longcat-flash
```
```python
from mlx_vlm import load, generate
model, processor = load("AlazarM/LongCat-Flash-Lite-Sparse-4bit", trust_remote_code=True)
tok = processor.tokenizer
text = tok.apply_chat_template(
    [{"role": "user", "content": "What is the capital of France?"}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, processor, text, max_tokens=64, temperature=0.0))
# -> The capital of France is Paris.
```

## Throughput (M5 Max, 128 GB, batch 1, greedy)

Same methodology across quantizations (chunk-512 prefill, warmed kernels).

**Decode tok/s**
| ctx | 4-bit | 6-bit | 8-bit |
|--:|--:|--:|--:|
| 512   | 112 | 87 | 80 |
| 1024  | 101 | 83 | 75 |
| 2048  |  85 | 72 | 65 |
| 4096  |  84 | 72 | 65 |
| 8192  |  83 | 71 | 65 |
| 16384 |  79 | 66 | 64 |
| 32768 |  73 | 64 | 60 |

**Prefill tok/s**
| ctx | 4-bit | 6-bit | 8-bit |
|--:|--:|--:|--:|
| 512   | 3142 | 2581 | 2421 |
| 2048  | 2386 | 2366 | 1923 |
| 8192  | 1756 | 1627 | 1312 |
| 32768 |  623 |  504 |  492 |

**Footprint** — peak memory across 512→32k: 4-bit ~39–45 GB · 6-bit ~56–63 GB · 8-bit ~74–80 GB.

LSA's dynamic sparse selection activates once the KV length exceeds `index_topk` (2048), keeping decode nearly flat (4-bit 112→73 tok/s to 32k). Batch-1 decode is partly weight-bandwidth-bound, so lower precision is faster; higher precision trades that for quality — only ~3B params are active per token, so quant error has little room to hide.

## License
MIT, inherited from the base model.