---
title: Index-Translate-35B-A3B-preview-GGUF
canonical_url: "https://www.modelscope.cn/models/IndexTeam/Index-Translate-35B-A3B-preview-GGUF"
md_url: "https://www.modelscope.cn/models/IndexTeam/Index-Translate-35B-A3B-preview-GGUF.md"
repository: IndexTeam/Index-Translate-35B-A3B-preview-GGUF
last_updated: 2026-10-10
license: apache-2.0
base_model:
  - IndexTeam/Index-Translate-35B-A3B-preview
base_model_relation: quantized
library_name:
  - gguf
downloads: 683
stars: 11
tags:
  - gguf
  - translation
  - index
---

# Index-Translate-35B-A3B-preview-GGUF

> Index-Translate-35B-A3B-preview-GGUF - IndexTeam 在 ModelScope 开源的模型。Index-Translate-35B-A3B-preview-GGUF

IndexTeam/Index-Translate-35B-A3B-preview-GGUF 是 ModelScope 魔搭社区上的机器学习模型，采用 apache-2.0 许可，基于 IndexTeam/Index-Translate-35B-A3B-preview 构建。

- **Repository**: IndexTeam/Index-Translate-35B-A3B-preview-GGUF
- **License**: apache-2.0
- **Base model**: IndexTeam/Index-Translate-35B-A3B-preview
- **Tags**: gguf, translation, index
- **Downloads**: 683
- **Stars**: 11
- **Last updated**: 2026-10-10

Source: https://www.modelscope.cn/models/IndexTeam/Index-Translate-35B-A3B-preview-GGUF

---

# Index-Translate-35B-A3B-preview-GGUF

Official GGUF conversion of [IndexTeam/Index-Translate-35B-A3B-preview](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview), part of the **Index-Translate** multilingual translation model family (150 languages, terminology/format-constrained translation, controlled dubbing translation, long-document translation).

- Technical report: [Index-Translate: A Multilingual Translation Model Family](https://arxiv.org/abs/2609.40181)
- Code: [github.com/bilibili/Index-Translate](https://github.com/bilibili/Index-Translate)
- Converted with [llama.cpp](https://github.com/ggml-org/llama.cpp) (master, 2026-10); static post-training quantization, all bit-widths in this single repository.

## Provided files

(sorted by size)

| File | Size | Notes |
|:-----|-----:|:------|
| [mmproj-Q8_0](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.mmproj-Q8_0.gguf) | 0.61 GB | multi-modal projector supplement |
| [mmproj-f16](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.mmproj-f16.gguf) | 0.90 GB | multi-modal projector supplement |
| [Q2_K](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q2_K.gguf) | 13.25 GB | 2-bit, significant quality loss |
| [Q3_K_S](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q3_K_S.gguf) | 15.55 GB | 3-bit, noticeable quality loss |
| [Q3_K_M](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q3_K_M.gguf) | 17.17 GB | 3-bit, noticeable quality loss |
| [Q3_K_L](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q3_K_L.gguf) | 18.55 GB | 3-bit, noticeable quality loss |
| [IQ4_XS](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.IQ4_XS.gguf) | 19.39 GB | 4-bit I-quant, small |
| [Q4_K_S](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q4_K_S.gguf) | 20.37 GB | 4-bit, slightly smaller/faster |
| [Q4_K_M](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q4_K_M.gguf) | 21.71 GB | 4-bit, good balance, recommended |
| [Q5_K_S](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q5_K_S.gguf) | 24.56 GB | 5-bit, low quality loss |
| [Q5_K_M](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q5_K_M.gguf) | 25.35 GB | 5-bit, low quality loss |
| [Q6_K](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q6_K.gguf) | 29.21 GB | 6-bit, very low quality loss |
| [Q8_0](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.Q8_0.gguf) | 37.80 GB | 8-bit, near-lossless |
| [f16](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF/resolve/main/Index-Translate-35B-A3B-preview.f16-00001-of-00002.gguf) | 71.07 GB | 16-bit, lossless conversion baseline, 2 shards |

## Multimodal Image Translation (`mmproj`)

This repository includes official native multi-modal projector weights (`*.mmproj-*.gguf`), enabling end-to-end image-to-text translation across 150 languages **without any external OCR tools**:

- `Index-Translate-35B-A3B-preview.mmproj-Q8_0.gguf`: 8-bit quantized projector, recommended for production and local use.
- `Index-Translate-35B-A3B-preview.mmproj-f16.gguf`: 16-bit lossless conversion baseline.

### 1. Launch `llama-server` with Vision Projector:

```bash
llama-server \
  -m Index-Translate-35B-A3B-preview.Q4_K_M.gguf \
  --mmproj Index-Translate-35B-A3B-preview.mmproj-Q8_0.gguf \
  -ngl 99 \
  -c 32768 \
  --port 8000 \
  --alias Index-Translate-35B-A3B-preview
```

### 2. Run Image Translation:

Once the server is running, perform image translation with the official script:
```bash
python inference/llm/translate.py --image screenshot.png --target zh -m Index-Translate-35B-A3B-preview
```

Or test it directly via the online web demo at [https://index-translate.bilibili.com/?p=/site/translate.html](https://index-translate.bilibili.com/?p=/site/translate.html). Full guide: [Multimodal Image Translation Guide](https://github.com/bilibili/Index-Translate/blob/main/docs/image_translate.md).

## Usage

```bash
# Run the recommended Q4_K_M directly:
llama serve -hf IndexTeam/Index-Translate-35B-A3B-preview-GGUF:Q4_K_M
llama cli -hf IndexTeam/Index-Translate-35B-A3B-preview-GGUF:Q4_K_M
```

Translation prompt format (greedy decoding, temperature=0 recommended; use the chat template with `enable_thinking: false`):

```text
请将以下文本翻译为{target-language}，直接输出翻译结果，不要进行任何解释。

{source-text}
```

```bash
# llama.cpp: run the recommended Q4_K_M with the chat template and thinking disabled
llama-cli -hf IndexTeam/Index-Translate-35B-A3B-preview-GGUF:Q4_K_M --jinja --chat-template-kwargs '{"enable_thinking":false}' --temp 0
```

## Prompting & constrained translation (instTrans)

Beyond plain translation, the models follow the **instTrans** constrained-translation format. The official client wraps requests into the canonical structure `【源文】<text>` + numbered `1. 【硬性要求】<hard constraints>` + `2. 【注意】<soft constraints>` + suffix instructions:

- **Hard constraints** (binary, must hold): strict terminology glossary enforcement (e.g. `碳纤维:carbon fiber, 抗裂缝:crack resistance`), and format/structure preservation for JSON/CSV/code/placeholders.
- **Soft constraints** (graded): tone & style adaptation (e.g. formal business-email register), domain/word-sense disambiguation (e.g. *plant* → 工厂 in an industrial context), cross-sentence consistency, LaTeX preservation.
- **Syllable-controlled translation** (dubbing): use the Index-Homura checkpoints ([IndexTeam/Index-Homura-2B/9B](https://huggingface.co/IndexTeam/Index-Homura-2B)), which strictly respect a target syllable budget and can be combined with glossaries.

Full prompt reference: [github.com/bilibili/Index-Translate](https://github.com/bilibili/Index-Translate) (Instruction Following section, `docs/prompts.md`, `inference/llm/cases/`).

See the [base model card](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview) for the full instTrans constrained-translation format and serving presets.

## Consistency validation

Before release, each quantization tier was validated on GPU (NVIDIA A100) against the F16 conversion: per-token KL divergence / RMS Δp via `llama-perplexity`, plus greedy-generation spot checks against the original BF16 weights (transformers reference). Generation outputs of Q4_K_M matched the reference almost verbatim.

*Converted and published by the Index team, 2026-10-03.*
