---
title: calme-2.2-llama3-70b
canonical_url: "https://www.modelscope.cn/models/QwenCollection/calme-2.2-llama3-70b"
md_url: "https://www.modelscope.cn/models/QwenCollection/calme-2.2-llama3-70b.md"
repository: QwenCollection/calme-2.2-llama3-70b
last_updated: 2024-11-21
license: llama3
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - llama
architectures:
  - LlamaForCausalLM
parameters: 70.6B
tensor_type:
  - BF16
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
language:
  - en
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 2195
stars: 0
tags:
  - axolotl
  - finetune
  - dpo
  - facebook
  - meta
  - pytorch
  - llama
  - llama-3
  - chatml
---

# calme-2.2-llama3-70b

> calme-2.2-llama3-70b - QwenCollection 在 ModelScope 开源的模型。MaziyarPanahi/calme-2.2-llama3-70b

QwenCollection/calme-2.2-llama3-70b 是 ModelScope 魔搭社区上的 70.6B 参数text-generation模型，采用 llama3 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: QwenCollection/calme-2.2-llama3-70b
- **License**: llama3
- **Tasks**: text-generation
- **Parameters**: 70.6B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Tags**: axolotl, finetune, dpo, facebook, meta, pytorch, llama, llama-3, chatml
- **Downloads**: 2195
- **Stars**: 0
- **Last updated**: 2024-11-21

Source: https://www.modelscope.cn/models/QwenCollection/calme-2.2-llama3-70b

---

<img src="./llama-3-merges.webp" alt="Llama-3 DPO Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>


# MaziyarPanahi/calme-2.2-llama3-70b

This model is a fine-tune (DPO) of `meta-llama/Meta-Llama-3-70B-Instruct` model.

PS: This fine-tuned model was previously known as `MaziyarPanahi/Llama-3-70B-Instruct-DPO-v0.2`. It was renamed to avoid any confusion with the original model.


# ⚡ Quantized GGUF

All GGUF models are available here: [MaziyarPanahi/calme-2.2-llama3-70b-GGUF](https://huggingface.co/MaziyarPanahi/calme-2.2-llama3-70b-GGUF)

# 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__calme-2.2-llama3-70b)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |37.98|
|IFEval (0-Shot)    |82.08|
|BBH (3-Shot)       |48.57|
|MATH Lvl 5 (4-Shot)|22.96|
|GPQA (0-shot)      |12.19|
|MuSR (0-shot)      |15.30|
|MMLU-PRO (5-shot)  |46.74|

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |78.96|
|AI2 Reasoning Challenge (25-Shot)|72.53|
|HellaSwag (10-Shot)              |86.22|
|MMLU (5-Shot)                    |80.41|
|TruthfulQA (0-shot)              |63.57|
|Winogrande (5-shot)              |82.79|
|GSM8k (5-shot)                   |88.25|

**Top 10 models on the Leaderboard**
<img src="./llama-qwen2-leaderboard.png" alt="Llama-3-70B finet-tuned models" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

# Prompt Template

This model uses `ChatML` prompt template:

```
<|im_start|>system
{System}
<|im_end|>
<|im_start|>user
{User}
<|im_end|>
<|im_start|>assistant
{Assistant}
````

# How to use

You can use this model by using `MaziyarPanahi/calme-2.2-llama3-70b` as the model name in Hugging Face's
transformers library.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
from transformers import pipeline
import torch

model_id = "MaziyarPanahi/calme-2.2-llama3-70b"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
    # attn_implementation="flash_attention_2"
)

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True
)

streamer = TextStreamer(tokenizer)

pipeline = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    model_kwargs={"torch_dtype": torch.bfloat16},
    streamer=streamer
)

# Then you can use the pipeline to generate text.

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|im_end|>"),
    tokenizer.convert_tokens_to_ids("<|eot_id|>") # safer to have this too
]

outputs = pipeline(
    prompt,
    max_new_tokens=2048,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
)
print(outputs[0]["generated_text"][len(prompt):])
```