---
title: AntAngelMed-INT4
canonical_url: "https://www.modelscope.cn/models/MedAIBase/AntAngelMed-INT4"
md_url: "https://www.modelscope.cn/models/MedAIBase/AntAngelMed-INT4.md"
repository: MedAIBase/AntAngelMed-INT4
last_updated: 2026-01-22
license: "Apache License 2.0"
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - bailing_moe_v2
architectures:
  - BailingMoeV2ForCausalLM
base_model:
  - MedAIBase/AntAngelMed
base_model_relation: quantized
library_name:
  - safetensors
  - pytorch
frameworks:
  - Pytorch
downloads: 493
stars: 5
---

# AntAngelMed-INT4

> AntAngelMed-INT4 - MedAIBase 在 ModelScope 开源的模型。MedAIBase/AntAngelMed-INT4 is a high-performance quantized version of MedAIBase/AntAngelMed designed for high-efficiency clinical applications. This model utilizes GPTQ INT4 quantization to significantly…

MedAIBase/AntAngelMed-INT4 是 ModelScope 魔搭社区上的text-generation模型，采用 Apache License 2.0 许可，基于 MedAIBase/AntAngelMed 构建。

- **Repository**: MedAIBase/AntAngelMed-INT4
- **License**: Apache License 2.0
- **Tasks**: text-generation
- **Base model**: MedAIBase/AntAngelMed
- **Downloads**: 493
- **Stars**: 5
- **Last updated**: 2026-01-22

Source: https://www.modelscope.cn/models/MedAIBase/AntAngelMed-INT4

---

## Model Overview

**MedAIBase/AntAngelMed-INT4** is a high-performance quantized version of [MedAIBase/AntAngelMed](https://modelscope.cn/models/MedAIBase/AntAngelMed) designed for high-efficiency clinical applications. This model utilizes **GPTQ INT4 quantization** to significantly accelerate inference speeds and reduce memory consumption while maintaining high numerical accuracy. It is specifically optimized for large-scale medical AI deployment.


## Model Size

| 数据精度 (Precision) | 权重大小 (Size) | 备注 |
|----------------------|----------------|------|
| BF16 (Brain Float 16) | 192 GB | 16 位浮点数 |
| FP8 (Float 8)         | 92 GB  | 8 位浮点数 |
| INT4 (Integer 4)      | 52 GB  | 4 位量化 |

## Performance & Acceleration

The FP8-quantized architecture is purpose-built for high-concurrency production environments. It addresses the "memory wall" often encountered in medical LLMs, enabling the deployment of larger models on cost-effective hardware.

These metrics demonstrate robust acceleration performance across diverse and complex domains.

![image](https://hackmd.io/_uploads/rJgdJaprWl.png)


## Model Accuracy

Despite the aggressive quantization, the model maintains high-fidelity outputs. As shown below, the accuracy trade-off is negligible, ensuring clinical reliability is preserved.

![image](https://hackmd.io/_uploads/rkAOy6aHWg.png)



## Quick Start

### Requirements

- H200-class Computational Performance
- CUDA 12.0+
- PyTorch 2.0+

### Installation

```bash
pip install sglang==0.5.6
```

### Inference with SGLang

```python
python3 -m sglang.launch_server  \
    --model-path MedAIBase/AntAngelMed-INT4 \
    --host 0.0.0.0 --port 30012  \
    --trust-remote-code  \
    --attention-backend fa3  \
    --mem-fraction-static 0.9 \
    --tp-size 1  

```