---
title: Llama3-Chinese-8B-Instruct
canonical_url: "https://www.modelscope.cn/models/FlagAlpha/Llama3-Chinese-8B-Instruct"
md_url: "https://www.modelscope.cn/models/FlagAlpha/Llama3-Chinese-8B-Instruct.md"
repository: FlagAlpha/Llama3-Chinese-8B-Instruct
last_updated: 2024-04-24
license: "Apache License 2.0"
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - llama
architectures:
  - LlamaForCausalLM
parameters: 8.0B
tensor_type:
  - F16
library_name:
  - pytorch
  - safetensors
frameworks:
  - pytorch
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 31836
stars: 47
---

# Llama3-Chinese-8B-Instruct

> Llama3-Chinese-8B-Instruct - FlagAlpha 在 ModelScope 开源的模型。Llama3-8B 中文微调模型

FlagAlpha/Llama3-Chinese-8B-Instruct 是 ModelScope 魔搭社区上的 8.0B 参数text-generation模型，采用 Apache License 2.0 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: FlagAlpha/Llama3-Chinese-8B-Instruct
- **License**: Apache License 2.0
- **Tasks**: text-generation
- **Parameters**: 8.0B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 31836
- **Stars**: 47
- **Last updated**: 2024-04-24

Source: https://www.modelscope.cn/models/FlagAlpha/Llama3-Chinese-8B-Instruct

---

### Clone with HTTP
```bash
git clone https://www.modelscope.cn/FlagAlpha/Llama3-Chinese-8B-Instruct.git
```

# Llama3-Chinese-8B 

Llama3-Chinese-8B基于Llama3-8B的中文对话模型，由Llama中文社区和AtomEcho（原子回声）联合研发，我们会持续提供更新的模型参数，模型训练过程见(https://llama.family)。

模型的部署、训练、微调等方法详见Llama中文社区GitHub仓库：https://github.com/LlamaFamily/Llama-Chinese

## 在线体验

https://llama.family/chat/#/

## 如何使用

下载模型
```
git clone https://www.modelscope.cn/FlagAlpha/Llama3-Chinese-8B-Instruct.git
```

使用
```
import transformers
import torch


model_id = "./Llama3-Chinese-8B-Instruct"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.float16},
    device="cuda",
)


messages = [{"role": "system", "content": ""}]

messages.append(
                {"role": "user", "content": "介绍一下机器学习"}
            )

prompt = pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
    )

terminators = [
        pipeline.tokenizer.eos_token_id,
        pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
    ]
outputs = pipeline(
    prompt,
    max_new_tokens=512,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9
)

content = outputs[0]["generated_text"][len(prompt):]

print(content)
```
