---
title: LFM2-8B-A1B-3bit-MLX
canonical_url: "https://www.modelscope.cn/models/mlx-community/LFM2-8B-A1B-3bit-MLX"
md_url: "https://www.modelscope.cn/models/mlx-community/LFM2-8B-A1B-3bit-MLX.md"
repository: mlx-community/LFM2-8B-A1B-3bit-MLX
last_updated: 2025-10-10
license: other
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - lfm2_moe
architectures:
  - Lfm2MoeForCausalLM
base_model:
  - LiquidAI/LFM2-8B-A1B
base_model_relation: quantized
parameters: 1.0B
tensor_type:
  - F32
  - U32
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 65
stars: 0
tags:
  - mlx
  - apple-silicon
  - liquidai
  - lfm2
  - moe
  - transformer
  - long-context
  - instruct
  - quantized
  - 3bit
  - coding
---

# LFM2-8B-A1B-3bit-MLX

> LFM2-8B-A1B-3bit-MLX - mlx-community 在 ModelScope 开源的模型。LFM2-8B-A1B — MLX 3-bit (Apple Silicon)

mlx-community/LFM2-8B-A1B-3bit-MLX 是 ModelScope 魔搭社区上的 1.0B 参数text-generation模型，采用 other 许可，基于 LiquidAI/LFM2-8B-A1B 构建。

- **Repository**: mlx-community/LFM2-8B-A1B-3bit-MLX
- **License**: other
- **Tasks**: text-generation
- **Parameters**: 1.0B
- **Base model**: LiquidAI/LFM2-8B-A1B
- **Tags**: mlx, apple-silicon, liquidai, lfm2, moe, transformer, long-context, instruct, quantized, 3bit, coding
- **Downloads**: 65
- **Stars**: 0
- **Last updated**: 2025-10-10

Source: https://www.modelscope.cn/models/mlx-community/LFM2-8B-A1B-3bit-MLX

---

# LFM2-8B-A1B — **MLX 3-bit** (Apple Silicon)

**Maintainer / Publisher:** [**Susant Achary**](https://huggingface.co/Susant-Achary)  
**Upstream model:** [LiquidAI/LFM2-8B-A1B](https://huggingface.co/LiquidAI/LFM2-8B-A1B)  
**This repo (MLX 3-bit):** `mlx-community/LFM2-8B-A1B-3bit-MLX`

This repository provides an **Apple-Silicon-optimized MLX build** of **LFM2-8B-A1B** at **3-bit** quantization.  
3-bit is an excellent **size↔quality sweet spot** on many Macs—very small memory footprint with surprisingly solid answer quality and snappy decoding.

---

## 🔎 What is LFM2-8B-A1B?

- **Architecture:** Mixture-of-Experts (**MoE**) Transformer.  
- **Size:** ~**8B total parameters** with ~**1B active** per token (the “A1B” naming commonly indicates ~1B active params).  
- **Why MoE?** Per token, only a subset of experts is activated → **lower compute per token** while retaining a larger parameter pool for expressivity.

> **Memory reality on a single device:** Even though ~1B parameters are *active* at a time, **all experts typically reside in memory** in single-device runs. Plan **RAM** based on **total parameters**, not just the active slice.

---

## 📦 What’s in this MLX build

- `config.json` (MLX), `mlx_model*.safetensors` (**3-bit** shards)  
- Tokenizer: `tokenizer.json`, `tokenizer_config.json`  
- Metadata: `model_index.json` (and/or processor metadata as applicable)

Target: **macOS** on **Apple Silicon (M-series)** using **Metal/MPS**.

---

## ✅ Intended use

- General **instruction following**, chat, and summarization  
- **RAG** back-ends and long-context assistants on device  
- **Schema-guided** structured outputs (JSON) where low RAM is a priority

## ⚠️ Limitations

- 3-bit is **lossy**: tiny improvements in latency/RAM come with some accuracy trade-off vs 6/8-bit.  
- For very long contexts and/or batching, **KV-cache** can dominate memory—tune `max_tokens` and batch size.  
- Add your own **guardrails/safety** for production deployments.

---

## 🔢 RAM planning (3-bit, MoE, MLX)

You asked to **assume and decide** realistic ranges. The numbers below are **practical starting points**—verify on your machine.

### Rule-of-thumb components

- **Weights (3-bit):** ≈ `total_params × 0.375 byte` → for **8B params ≈ ~3.0 GB**  
- **Runtime overhead:** MLX graph/tensors/metadata → **~0.6–1.0 GB**  
- **KV-cache:** grows with **context × layers × heads × dtype** → **~0.8–2.5+ GB**

### Indicative peak RAM (batch=1)

| Context window | Estimated peak RAM |
|---|---:|
| **4k tokens**  | **~4.4–5.5 GB** |
| **8k tokens**  | **~5.2–6.6 GB** |
| **16k tokens** | **~6.5–8.8 GB** |

> For ≤2k windows you may see **~4.0–4.8 GB**. Larger windows/batches increase KV-cache and peak RAM.

---

## 🧭 Precision choices for LFM2-8B-A1B (lineup planning)

While this card is **3-bit**, teams often publish multiple precisions. Use this table as a **planning guide** (8B MoE LM; actuals depend on context/batch/prompts):

| Variant | Typical Peak RAM | Relative Speed | Typical Behavior | When to choose |
|---|---:|:---:|---|---|
| **3-bit** *(this repo)* | **~4.4–8.8 GB** | **🔥🔥🔥🔥** | **Direct, concise**, great latency | **Default** on 8–16 GB Macs |
| **6-bit** | ~7.5–12.5 GB | 🔥🔥 | Best quality under quant | Choose if RAM allows |
| **8-bit** | ~9.5–12+ GB | 🔥🔥 | Largest quantized size / highest fidelity | When you prefer simpler 8-bit workflows |

> **MoE caveat:** MoE lowers **compute per token**; unless experts are **paged/partitioned**, **memory** still scales with **total parameters** on a single device.

---

## 🚀 Quickstart (CLI — MLX)

**Deterministic generation**
```bash
python -m mlx_lm.generate \
  --model mlx-community/LFM2-8B-A1B-3bit-MLX \
  --prompt "Summarize the following in 5 concise bullet points:\n<your text>" \
  --max-tokens 256 \
  --temperature 0.0 \
  --device mps \
  --seed 0