---
title: Phi-3-mini-4k-instruct_fp16
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/Phi-3-mini-4k-instruct_fp16"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/Phi-3-mini-4k-instruct_fp16.md"
repository: AI-ModelScope/Phi-3-mini-4k-instruct_fp16
last_updated: 2024-06-25
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - phi3
architectures:
  - Phi3ForCausalLM
library_name:
  - onnx
  - pytorch
frameworks:
  - Pytorch
inference_backends:
  - "deploy_task text/emb"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 17
stars: 0
tags:
  - ONNX
  - DML
  - ONNXRuntime
  - nlp
  - conversational
---

# Phi-3-mini-4k-instruct_fp16

> Phi-3-mini-4k-instruct_fp16 - AI-ModelScope 在 ModelScope 开源的模型。Phi-3 Mini-4K-Instruct ONNX model for onnxruntime-web This is the same models as the official phi3 onnx model with a few changes to make it work for onnxruntime-web:

AI-ModelScope/Phi-3-mini-4k-instruct_fp16 是 ModelScope 魔搭社区上的text-generation模型，采用 mit 许可，可用 deploy_task text/emb、sglang 0.5.2、vllm 0.9.2 部署。

- **Repository**: AI-ModelScope/Phi-3-mini-4k-instruct_fp16
- **License**: mit
- **Tasks**: text-generation
- **Inference backends**: deploy_task text/emb, sglang 0.5.2, vllm 0.9.2
- **Tags**: ONNX, DML, ONNXRuntime, nlp, conversational
- **Downloads**: 17
- **Stars**: 0
- **Last updated**: 2024-06-25

Source: https://www.modelscope.cn/models/AI-ModelScope/Phi-3-mini-4k-instruct_fp16

---

# Phi-3 Mini-4K-Instruct ONNX model for onnxruntime-web
This is the same models as the [official phi3 onnx model](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx) with a few changes to make it work for onnxruntime-web:

1. the model is fp16 with int4 block quantization for weights
2. the 'logits' output is fp32 
3. the model uses MHA instead of GQA
4. onnx and external data file need to stay below 2GB to be cacheable in chromium