---
title: TinyLlama-1.1B-Chat-v1.0-autoround-4bit
canonical_url: "https://www.modelscope.cn/models/LLM-Research/TinyLlama-1.1B-Chat-v1.0-autoround-4bit"
md_url: "https://www.modelscope.cn/models/LLM-Research/TinyLlama-1.1B-Chat-v1.0-autoround-4bit.md"
repository: LLM-Research/TinyLlama-1.1B-Chat-v1.0-autoround-4bit
last_updated: 2024-07-26
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - llama
architectures:
  - LlamaForCausalLM
parameters: 261.2M
tensor_type:
  - BF16
  - I32
  - F16
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 4115
stars: 0
---

# TinyLlama-1.1B-Chat-v1.0-autoround-4bit

> TinyLlama-1.1B-Chat-v1.0-autoround-4bit - LLM-Research 在 ModelScope 开源的模型。This model has been quantized using GPTQModel.

LLM-Research/TinyLlama-1.1B-Chat-v1.0-autoround-4bit 是 ModelScope 魔搭社区上的 261.2M 参数text-generation模型，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: LLM-Research/TinyLlama-1.1B-Chat-v1.0-autoround-4bit
- **Tasks**: text-generation
- **Parameters**: 261.2M
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 4115
- **Stars**: 0
- **Last updated**: 2024-07-26

Source: https://www.modelscope.cn/models/LLM-Research/TinyLlama-1.1B-Chat-v1.0-autoround-4bit

---

This model has been quantized using [GPTQModel](https://github.com/ModelCloud/GPTQModel).

- **Bits**: 4
- **Group Size**: 128
- **Desc Act**: true
- **Static Groups**: false
- **Sym**: true
- **LM Head**: false
- **Damp Percent**: 0.01
- **True Sequential**: true
- **Model Name or Path**: 
- **Model File Base Name**: model
- **Quant Method**: auto_round
- **Checkpoint Format**: gptq
- **Metadata**
  - **Quantizer**: gptqmodel:0.9.8-dev0
  - **Enable Full Range**: false
  - **Batch Size**: 1
  - **AMP**: true
  - **LR Scheduler**: null
  - **Enable Quanted Input**: true
  - **Enable Minmax Tuning**: true
  - **Learning Rate (LR)**: null
  - **Minmax LR**: null
  - **Low GPU Memory Usage**: true
  - **Iterations (Iters)**: 200
  - **Sequence Length (Seqlen)**: 2048
  - **Number of Samples (Nsamples)**: 512
  - **Sampler**: rand
  - **Seed**: 42
  - **Number of Blocks (Nblocks)**: 1
  - **Gradient Accumulate Steps**: 1
  - **Not Use Best MSE**: false
  - **Dynamic Max Gap**: -1
  - **Data Type**: int
  - **Scale Data Type (Scale Dtype)**: fp16