---
title: Index-Translate-35B-A3B-preview-FP4
canonical_url: "https://www.modelscope.cn/models/IndexTeam/Index-Translate-35B-A3B-preview-FP4"
md_url: "https://www.modelscope.cn/models/IndexTeam/Index-Translate-35B-A3B-preview-FP4.md"
repository: IndexTeam/Index-Translate-35B-A3B-preview-FP4
last_updated: 2026-10-04
license: apache-2.0
model_type:
  - qwen3_5_moe
architectures:
  - Qwen3_5MoeForConditionalGeneration
base_model:
  - IndexTeam/Index-Translate-35B-A3B-preview
base_model_relation: quantized
parameters: 18.8B
tensor_type:
  - F32
  - U8
  - F8_E4M3
  - BF16
library_name:
  - safetensors
downloads: 453
stars: 1
tags:
  - nvfp4
  - fp4
  - modelopt
  - translation
  - index
---

# Index-Translate-35B-A3B-preview-FP4

> Index-Translate-35B-A3B-preview-FP4 - IndexTeam 在 ModelScope 开源的模型。Index-Translate-35B-A3B-preview-FP4

IndexTeam/Index-Translate-35B-A3B-preview-FP4 是 ModelScope 魔搭社区上的 18.8B 参数机器学习模型，采用 apache-2.0 许可，基于 IndexTeam/Index-Translate-35B-A3B-preview 构建。

- **Repository**: IndexTeam/Index-Translate-35B-A3B-preview-FP4
- **License**: apache-2.0
- **Parameters**: 18.8B
- **Base model**: IndexTeam/Index-Translate-35B-A3B-preview
- **Tags**: nvfp4, fp4, modelopt, translation, index
- **Downloads**: 453
- **Stars**: 1
- **Last updated**: 2026-10-04

Source: https://www.modelscope.cn/models/IndexTeam/Index-Translate-35B-A3B-preview-FP4

---

# Index-Translate-35B-A3B-preview-FP4

Official NVFP4 (W4A4) quantization of [IndexTeam/Index-Translate-35B-A3B-preview](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview), part of the **Index-Translate** multilingual translation model family (150 languages, terminology/format-constrained translation, controlled dubbing translation, long-document translation).

- Technical report: [Index-Translate: A Multilingual Translation Model Family](https://arxiv.org/abs/2609.40181)
- Code: [github.com/bilibili/Index-Translate](https://github.com/bilibili/Index-Translate)

## Quantization

- Scheme: `NVFP4` (W4A4: 4-bit floating-point weights with per-group-16 scales, 4-bit floating-point activations with calibrated per-tensor global scales), produced with [NVIDIA Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) 0.44 (`modelopt` format, see `hf_quant_config.json`). Calibrated on a small bilingual translation corpus.
- All `Linear` layers of the language model are quantized; `lm_head`, embeddings, MoE router gates and shared-expert gates are kept in BF16. MTP (multi-token prediction) weights are preserved.
- Format: ModelOpt-exported safetensors with `hf_quant_config.json` - serve with vLLM (`--quantization modelopt`, auto-detected in recent vLLM).

## Consistency validation

Measured on the quantized model in memory on an NVIDIA A100 before export against the original BF16 checkpoint (greedy decoding, official translation prompt):

| Metric | BF16 | FP4 | Delta |
|:-------|-----:|----:|------:|
| Perplexity (fixed corpus) | 2.5621 | 2.6039 | +1.63% |
| zh->en generation identical | - | - | yes |
| en->zh generation identical | - | - | yes |

## Usage

```bash
vllm serve IndexTeam/Index-Translate-35B-A3B-preview-FP4 --host 127.0.0.1 --port 8000 --max-model-len 4096
```

**Hardware note:** full NVFP4 (W4A4) acceleration requires an NVIDIA Blackwell GPU (SM100+, e.g. B200 / RTX 50 series). On older GPUs (Hopper / Ampere) vLLM loads the checkpoint with weight-only dequantization - memory is still reduced, but there is no FP4 compute speedup. For non-Blackwell serving we recommend the [FP8 build](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-FP8).

Translation prompt format (greedy decoding, temperature=0 recommended; chat template with `enable_thinking: false`):

```text
请将以下文本翻译为{target-language}，直接输出翻译结果，不要进行任何解释。

{source-text}
```

## Prompting & constrained translation (instTrans)

Beyond plain translation, the models follow the **instTrans** constrained-translation format. The official client wraps requests into the canonical structure `【源文】<text>` + numbered `1. 【硬性要求】<hard constraints>` + `2. 【注意】<soft constraints>` + suffix instructions:

- **Hard constraints** (binary, must hold): strict terminology glossary enforcement (e.g. `碳纤维:carbon fiber, 抗裂缝:crack resistance`), and format/structure preservation for JSON/CSV/code/placeholders.
- **Soft constraints** (graded): tone & style adaptation (e.g. formal business-email register), domain/word-sense disambiguation (e.g. *plant* -> 工厂 in an industrial context), cross-sentence consistency, LaTeX preservation.
- **Syllable-controlled translation** (dubbing): the Index-Homura checkpoints ([IndexTeam/Index-Homura-2B](https://huggingface.co/IndexTeam/Index-Homura-2B), [IndexTeam/Index-Homura-9B](https://huggingface.co/IndexTeam/Index-Homura-9B)) strictly respect a target syllable budget and can be combined with glossaries.

Full prompt reference: [github.com/bilibili/Index-Translate](https://github.com/bilibili/Index-Translate) (Instruction Following section, `docs/prompts.md`, `inference/llm/cases/`).

See the [base model card](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview) for the full instTrans constrained-translation format and serving presets. GGUF builds for local inference are published in [Index-Translate-35B-A3B-preview-GGUF](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-GGUF), and an FP8 build for Hopper/Ampere serving in [Index-Translate-35B-A3B-preview-FP8](https://huggingface.co/IndexTeam/Index-Translate-35B-A3B-preview-FP8).

*Quantized and published by the Index team, 2026-10-04.*
