---
title: AntAngelMed-FP8
canonical_url: "https://www.modelscope.cn/models/MedAIBase/AntAngelMed-FP8"
md_url: "https://www.modelscope.cn/models/MedAIBase/AntAngelMed-FP8.md"
repository: MedAIBase/AntAngelMed-FP8
last_updated: 2026-01-04
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - bailing_moe_v2
architectures:
  - BailingMoeV2ForCausalLM
library_name:
  - safetensors
  - pytorch
frameworks:
  - Pytorch
downloads: 881
stars: 10
---

# AntAngelMed-FP8

> AntAngelMed-FP8 - MedAIBase 在 ModelScope 开源的模型。MedAIBase/AntAngelMed-FP8 is a high-performance quantized version of MedAIBase/AntAngelMed designed for high-efficiency clinical applications. This model utilizes Blockwise FP8 quantization to significantly…

MedAIBase/AntAngelMed-FP8 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可。

- **Repository**: MedAIBase/AntAngelMed-FP8
- **License**: apache-2.0
- **Tasks**: text-generation
- **Downloads**: 881
- **Stars**: 10
- **Last updated**: 2026-01-04

Source: https://www.modelscope.cn/models/MedAIBase/AntAngelMed-FP8

---

# AntAngelMed-FP8

## Model Overview

**MedAIBase/AntAngelMed-FP8** is a high-performance quantized version of [MedAIBase/AntAngelMed](https://huggingface.co/MedAIBase/AntAngelMed) designed for high-efficiency clinical applications. This model utilizes **Blockwise FP8 quantization** to significantly accelerate inference speeds and reduce memory consumption while maintaining high numerical accuracy. It is specifically optimized for large-scale medical AI deployment.


## Performance & Acceleration

The FP8-quantized architecture is purpose-built for high-concurrency production environments. It addresses the "memory wall" often encountered in medical LLMs, enabling the deployment of larger models on cost-effective hardware.

These metrics demonstrate robust acceleration performance across diverse and complex domains.


![1](https://hackmd.io/_uploads/S1pbAzWEbg.png)
![2](https://hackmd.io/_uploads/H1H9DNWNWl.png)


## Model Accuracy
Despite the aggressive quantization, the model maintains high-fidelity outputs. As shown below, the accuracy trade-off is negligible, ensuring clinical reliability is preserved.

![3](https://hackmd.io/_uploads/S1oMYS-N-g.png)


## Quick Start

### Requirements

- H200-class Computational Performance
- CUDA 12.0+
- PyTorch 2.0+

### Installation

```bash
pip install sglang==0.5.6
```

### Inference with SGLang

```python
python3 -m sglang.launch_server  \
    --model-path MedAIBase/AntAngelMed-FP8 \
    --host 0.0.0.0 --port 30012  \
    --trust-remote-code  \
    --attention-backend fa3  \
    --mem-fraction-static 0.9 \
    --tp-size 1  \
```