---
title: Hy3-SF-INT8
canonical_url: "https://www.modelscope.cn/models/siliconflow/Hy3-SF-INT8"
md_url: "https://www.modelscope.cn/models/siliconflow/Hy3-SF-INT8.md"
repository: siliconflow/Hy3-SF-INT8
last_updated: 2026-07-22
license: "Apache License 2.0"
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - hy_v3
architectures:
  - HYV3ForCausalLM
parameters: 300.0B
tensor_type:
  - I8
  - F32
  - BF16
library_name:
  - safetensors
  - pytorch
frameworks:
  - Pytorch
downloads: 2463
stars: 1
tags:
  - int8
  - w8a8
  - moe
  - mtp
---

# Hy3-SF-INT8

> Hy3-SF-INT8 - siliconflow 在 ModelScope 开源的模型。Rotated W8A8 INT8 Hy3 checkpoint with MTP, quantized by SiliconFlow CrossingBits.

siliconflow/Hy3-SF-INT8 是 ModelScope 魔搭社区上的 300.0B 参数text-generation模型，采用 Apache License 2.0 许可。

- **Repository**: siliconflow/Hy3-SF-INT8
- **License**: Apache License 2.0
- **Tasks**: text-generation
- **Parameters**: 300.0B
- **Tags**: int8, w8a8, moe, mtp
- **Downloads**: 2463
- **Stars**: 1
- **Last updated**: 2026-07-22

Source: https://www.modelscope.cn/models/siliconflow/Hy3-SF-INT8

---

# Hy3-SF-INT8

Hy3-SF-INT8 is a rotated W8A8 INT8 checkpoint based on [Tencent-Hunyuan/Hy3-FP8](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-FP8), published by SiliconFlow for efficient text generation and deployment.

## Model information

- **Base model:** Tencent Hy3
- **Architecture:** 295B Mixture-of-Experts, 21B active parameters
- **Task:** Text generation
- **Context length:** 256K
- **Languages:** Chinese and English
- **Weight format:** Safetensors
- **License:** Apache License 2.0

The tokenizer, chat template, generation configuration, rotation matrix, and one MTP layer are included in this repository.

## Quantization

The checkpoint was produced with [CrossingBits](https://github.com/SiliconFlow/crossingbits):

| Item | Setting |
|---|---|
| Weight / activation dtype | INT8 / INT8 |
| Granularity | Per-channel |
| Recipe | Rotation |
| Router | BF16, excluded from quantization to preserve expert selection |
| Main LM head | BF16 |
| MTP | Included and quantized |

This is a CrossingBits checkpoint. Native Hy3 Hugging Face, vLLM, and SGLang loaders do not consume its internal W8A8 layout directly.

## Verification

The saved artifact passed conversion-aware structural, numerical, and resident inference checks:

- **Manifest:** 187,087 output tensors; 0 missing keys, unexplained extras, or shape/dtype mismatches.
- **Shard integrity:** all 99 safetensors shards match `model.safetensors.index.json`; no missing or orphan shards.
- **All-zero sweep:** 187,087 tensors checked with 0 corruption failures. The only all-zero output tensor is the MTP `expert_bias`, whose canonical source is also all zero.
- **Canonical W8A8 samples:** eight weights spanning early, middle, late, and MTP layers achieved cosine similarity `0.999962–0.999964` and relative error `0.847%–0.870%` against provider-FP8-dequantized, fused, rotated references.
- **Excluded state:** all 80 BF16 routers passed forward-equivalence checks; 242 unchanged tensors matched exactly; 166 fused norm tensors were exact identity values.
- **Resident main decode:** 273.8 GiB INT8 + 2.0 GiB BF16 + 0.8 GiB FP32 loaded across four H20 GPUs without CPU/disk offload. Prompt `What is the capital of France?` produced `The capital of France is Paris.`
- **MTP acceptance:** the official five-sample evaluation covered 552 positions and measured Top-1 acceptance `0.5217` and Top-5 acceptance `0.7029`.

The full-run calibration report measured KLD `0.0801` and rejection rate `0.0351`.

## Download

```python
from modelscope import snapshot_download

model_dir = snapshot_download("siliconflow/Hy3-SF-INT8")
```

Or clone it with Git LFS:

```bash
git lfs install
git clone https://www.modelscope.cn/siliconflow/Hy3-SF-INT8.git
```

## Usage

Use the CrossingBits loader or an inference runtime with explicit support for the CrossingBits Hy3 W8A8 format. A minimal resident smoke test is:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 USE_MTP=0 python -m crossingbits.tools.tiny_infer \
  --model /path/to/Hy3-SF-INT8 \
  --prompt "What is the capital of France?" \
  --max-new-tokens 16 --dtype bf16
```

Evaluate the included MTP layer with:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 USE_MTP=1 python evals/eval_mtp_acceptance.py \
  --model /path/to/Hy3-SF-INT8 --dtype bf16 --top-k 5
```

Use the included tokenizer and chat template for chat applications. Model capabilities and limitations generally follow the Hy3 base model; evaluate quality and performance for your target workload before production deployment.

## License

This checkpoint follows the Apache License 2.0 terms of the Hy3 base model. See `LICENSE` for details.