---
title: InternVL3-8B-MLX-4bit
canonical_url: "https://www.modelscope.cn/models/mlx-community/InternVL3-8B-MLX-4bit"
md_url: "https://www.modelscope.cn/models/mlx-community/InternVL3-8B-MLX-4bit.md"
repository: mlx-community/InternVL3-8B-MLX-4bit
last_updated: 2026-02-23
license: other
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
model_type:
  - internvl_chat
architectures:
  - InternVLChatModel
base_model:
  - mlx-community/InternVL3-8B-bf16
base_model_relation: quantized
library_name:
  - mlx
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - multilingual
inference_backends:
  - "deploy_task vlm/text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 20
stars: 0
tags:
  - mlx
  - mlx-vlm
  - internvl
  - internvl3
  - 4-bit
  - quantized
  - vision-language-model
  - apple-silicon
---

# InternVL3-8B-MLX-4bit

> InternVL3-8B-MLX-4bit - mlx-community 在 ModelScope 开源的模型。InternVL3-8B-MLX-4bit

mlx-community/InternVL3-8B-MLX-4bit 是 ModelScope 魔搭社区上的image-text-to-text模型，采用 other 许可，基于 mlx-community/InternVL3-8B-bf16 构建，可用 deploy_task vlm/text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: mlx-community/InternVL3-8B-MLX-4bit
- **License**: other
- **Tasks**: image-text-to-text
- **Base model**: mlx-community/InternVL3-8B-bf16
- **Inference backends**: deploy_task vlm/text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Tags**: mlx, mlx-vlm, internvl, internvl3, 4-bit, quantized, vision-language-model, apple-silicon
- **Downloads**: 20
- **Stars**: 0
- **Last updated**: 2026-02-23

Source: https://www.modelscope.cn/models/mlx-community/InternVL3-8B-MLX-4bit

---

# InternVL3-8B-MLX-4bit

This repository contains a 4-bit MLX quantized conversion of `mlx-community/InternVL3-8B-bf16` for Apple Silicon inference.

## Conversion Details

| Setting | Value |
| --- | --- |
| Source model | `mlx-community/InternVL3-8B-bf16` |
| Conversion tool | `mlx_vlm.convert` |
| Quantization bits | `4` |
| Group size | `64` |
| Quantization mode | `affine` |
| Quant predicate | none (uniform quantization) |

Conversion command used:

```bash
python3 -m mlx_vlm convert \
  --hf-path "mlx-community/InternVL3-8B-bf16" \
  --mlx-path "./models/InternVL3-8B-4bit" \
  -q --q-bits 4 --q-group-size 64
```

## Validation

| Test | Status |
| --- | --- |
| Text generation load test | passed |

Verification command:

```bash
python3 -m mlx_vlm generate \
  --model "./models/InternVL3-8B-4bit" \
  --prompt "Reply with exactly: OK" \
  --max-tokens 8 --temperature 0
```

Observed response: `OK`

## Usage

Install:

```bash
python3 -m pip install -U mlx-vlm
```

Run locally from this folder:

```bash
python3 -m mlx_vlm generate \
  --model "." \
  --prompt "Describe the image briefly." \
  --image path/to/image.jpg \
  --max-tokens 256 \
  --temperature 0
```

Run from Hugging Face after upload:

```bash
python3 -m mlx_vlm generate \
  --model "mlx-community/InternVL3-8B-MLX-4bit" \
  --prompt "Describe the image briefly." \
  --image path/to/image.jpg \
  --max-tokens 256 \
  --temperature 0
```

## Notes

- This conversion does not upload anything automatically.
- Quantization changes numerical behavior relative to bf16 weights.
- During local tests, `mlx_vlm` emitted an upstream tokenizer regex warning from the source model assets.

## Links

- Source model: https://huggingface.co/mlx-community/InternVL3-8B-bf16
- MLX: https://github.com/ml-explore/mlx
- mlx-vlm: https://github.com/Blaizzy/mlx-vlm

## License

Follows the upstream model license terms from the source repository.