---
title: Sehyo-Qwen3.5-35B-A3B-NVFP4
canonical_url: "https://www.modelscope.cn/models/hf/Sehyo-Qwen3.5-35B-A3B-NVFP4"
md_url: "https://www.modelscope.cn/models/hf/Sehyo-Qwen3.5-35B-A3B-NVFP4.md"
repository: hf/Sehyo-Qwen3.5-35B-A3B-NVFP4
last_updated: 2026-03-02
model_type:
  - qwen3_5_moe
architectures:
  - Qwen3_5MoeForConditionalGeneration
base_model:
  - Qwen/Qwen3.5-35B-A3B
base_model_relation: quantized
parameters: 21.7B
tensor_type:
  - U8
  - BF16
  - F32
  - F8_E4M3
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 3610
stars: 1
tags:
  - quantized
  - nvfp4
  - compressed-tensors
  - llm-compressor
  - moe
  - qwen3.5
---

# Sehyo-Qwen3.5-35B-A3B-NVFP4

> Sehyo-Qwen3.5-35B-A3B-NVFP4 - hf 在 ModelScope 开源的模型。Qwen3.5-35B-A3B-NVFP4

hf/Sehyo-Qwen3.5-35B-A3B-NVFP4 是 ModelScope 魔搭社区上的 21.7B 参数机器学习模型，基于 Qwen/Qwen3.5-35B-A3B 构建。

- **Repository**: hf/Sehyo-Qwen3.5-35B-A3B-NVFP4
- **Parameters**: 21.7B
- **Base model**: Qwen/Qwen3.5-35B-A3B
- **Tags**: quantized, nvfp4, compressed-tensors, llm-compressor, moe, qwen3.5
- **Downloads**: 3610
- **Stars**: 1
- **Last updated**: 2026-03-02

Source: https://www.modelscope.cn/models/hf/Sehyo-Qwen3.5-35B-A3B-NVFP4

---

# Qwen3.5-35B-A3B-NVFP4

This is a quantized version of [Qwen/Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) using the **NVFP4** quantization scheme.

Please use nightly vLLM for support.

## Changelog
- **02/03/2026**: Re-quantized with MTP (multi-token prediction) weights preserved, enabling speculative decoding with vLLM.
- **25/02/2026**: Initial upload.

## Calibration

- **Samples**: 512 (256 from each dataset)
- **Datasets**:
  - [HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) (`train_sft` split)
  - [nvidia/Nemotron-Post-Training-Dataset-v2](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2) (`chat` split)
- **Max sequence length**: 4096
- **All experts calibrated**: `moe_calibrate_all_experts=True`

## Creation

This model was created using [VLLM's LLM Compressor](https://github.com/vllm-project/llm-compressor) with Qwen3.5 MoE support added via [PR #2383](https://github.com/vllm-project/llm-compressor/pull/2383). The PR adds a custom `CalibrationQwen3MoeSparseMoeBlock` that routes calibration data to all experts during quantization, ensuring every expert receives proper calibration for accurate NVFP4 quantization.