---
title: glm-4-9b-chat-1m-int4
canonical_url: "https://www.modelscope.cn/models/cjc1887415157/glm-4-9b-chat-1m-int4"
md_url: "https://www.modelscope.cn/models/cjc1887415157/glm-4-9b-chat-1m-int4.md"
repository: cjc1887415157/glm-4-9b-chat-1m-int4
last_updated: 2024-06-06
license: other
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - chatglm
architectures:
  - ChatGLMForConditionalGeneration
parameters: 5.5B
tensor_type:
  - U8
  - F32
  - F16
  - BF16
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - Pytorch
language:
  - zh
  - en
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy_turbomind 0.9.1"
  - "vllm 0.9.2"
downloads: 423
stars: 3
tags:
  - glm
  - chatglm
  - thudm
---

# glm-4-9b-chat-1m-int4

> glm-4-9b-chat-1m-int4 - cjc1887415157 在 ModelScope 开源的模型。glm4-9b-chat1m int4 quantization model files

cjc1887415157/glm-4-9b-chat-1m-int4 是 ModelScope 魔搭社区上的 5.5B 参数text-generation模型，采用 other 许可，可用 deploy_task text/emb、lmdeploy_turbomind 0.9.1、vllm 0.9.2 部署。

- **Repository**: cjc1887415157/glm-4-9b-chat-1m-int4
- **License**: other
- **Tasks**: text-generation
- **Parameters**: 5.5B
- **Inference backends**: deploy_task text/emb, lmdeploy_turbomind 0.9.1, vllm 0.9.2
- **Tags**: glm, chatglm, thudm
- **Downloads**: 423
- **Stars**: 3
- **Last updated**: 2024-06-06

Source: https://www.modelscope.cn/models/cjc1887415157/glm-4-9b-chat-1m-int4

---

# GLM-4-9B-Chat-1M-int4

**GLM-4-9B-Chat-1M int4量化的模型文件，支持`1M`上下文长度。**

## 运行模型

gradio demo:

```python
import torch
# from transformers import AutoModelForCausalLM, AutoTokenizer
from modelscope import AutoModelForCausalLM, AutoTokenizer
import gradio

device = "cuda"

tokenizer = AutoTokenizer.from_pretrained(r".\glm-4-9b-chat-1m-int4",trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    r".\glm-4-9b-chat-1m-int4",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True
)

def generate(query, history):
    with torch.no_grad():
        for response, _ in model.stream_chat(tokenizer, query):
            yield response

gradio.ChatInterface(generate).launch(inbrowser=True)
```