---
title: Qwen2-7B-Instruct
canonical_url: "https://www.modelscope.cn/models/rubraAI/Qwen2-7B-Instruct"
md_url: "https://www.modelscope.cn/models/rubraAI/Qwen2-7B-Instruct.md"
repository: rubraAI/Qwen2-7B-Instruct
last_updated: 2024-07-15
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - qwen2
architectures:
  - Qwen2ForCausalLM
parameters: 8.5B
tensor_type:
  - F32
  - BF16
library_name:
  - pytorch
  - safetensors
language:
  - en
  - zh
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 299
stars: 4
tags:
  - function-calling
  - tool-calling
  - agentic
  - rubra
  - conversational
  - "对话"
  - "工具调用"
  - "量化"
---

# Qwen2-7B-Instruct

> Qwen2-7B-Instruct - rubraAI 在 ModelScope 开源的模型。The model is the result of further post-training [Qwen/Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct). It is capable of complex multi-turn tool/function calling.

rubraAI/Qwen2-7B-Instruct 是 ModelScope 魔搭社区上的 8.5B 参数text-generation模型，采用 apache-2.0 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: rubraAI/Qwen2-7B-Instruct
- **License**: apache-2.0
- **Tasks**: text-generation
- **Parameters**: 8.5B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Tags**: function-calling, tool-calling, agentic, rubra, conversational, 对话, 工具调用, 量化
- **Downloads**: 299
- **Stars**: 4
- **Last updated**: 2024-07-15

Source: https://www.modelscope.cn/models/rubraAI/Qwen2-7B-Instruct

---

# Qwen2 7B Instruct

## Model description
本模型是在进一步微调 [Qwen/Qwen2-7B-Instruct](https://www.modelscope.cn/models/qwen/Qwen2-7B-Instruct) 的结果. 它能够进行多轮复杂的工具调用。

## Training

本模型在包含多样化工具的调用、聊天数据集和Instruct数据的专有数据集上进行了进一步post-training（freeze tuned和DPO）。

## 使用方法

您可以使用Hugging Face的`transformers`和rubra库[rubra-tools](https://github.com/rubra-ai/rubra-tools)来使用该模型，如下所示：

```
pip install rubra_tools torch==2.3.0 transformers accelerate modelscope
```

您还需要安装Node.js和npm。安装后，请安装`jsonrepair`包 - 用于修复模型的一些罕见错误。

```
npm install jsonrepair
```

### 1. 加载模型
```python
from modelscope import AutoModelForCausalLM, AutoTokenizer
import torch
from rubra_tools import preprocess_input, postprocess_output

model_id = "rubraAI/Meta-Llama-3-8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)
```

### 2. 定义函数
这里我们使用4个函数来解决一个简单的数学问题：
```python
functions = [
    {
            'type': 'function',
            'function': {
                'name': 'addition',
                'description': "Adds two numbers together",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to add',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Second number to add',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
        {
            'type': 'function',
            'function': {
                'name': 'subtraction',
                'description': "Subtracts two numbers",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to be subtracted from',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Number to subtract',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
        {
            'type': 'function',
            'function': {
                'name': 'multiplication',
                'description': "Multiply two numbers together",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to multiply',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Second number to multiply',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
        {
            'type': 'function',
            'function': {
                'name': 'division',
                'description': "Divide two numbers",
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'a': {
                            'description': 'First number to use as the dividend',
                            'type': 'string'
                        },
                        'b': {
                            'description': 'Second number to use as the divisor',
                            'type': 'string'
                        }
                    },
                    'required': []
                }
            }
        },
]
```

### 3. 开始对话
```python
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the result of four plus six? Take the result and add 2? Then multiply by 5 and then divide by two"},
]

def run_model(messages, functions):
    ## Format messages in Rubra's format
    formatted_msgs = preprocess_input(msgs=messages, tools=functions)

    text = tokenizer.apply_chat_template(
        formatted_msgs,
        tokenize=False,
        add_generation_prompt=True
    )
    model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

    generated_ids = model.generate(
        model_inputs.input_ids,
        max_new_tokens=512
    )
    generated_ids = [
        output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
    ]

    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    return response

raw_output = run_model(messages, functions)
# Check if there's a function call
function_call = postprocess_output(raw_output)
if function_call:
    print(function_call)
else:
    print(raw_output)
```

您应该会看到以下输出，这是AI助手进行的工具调用：
```
[{'id': 'fc65a533', 'function': {'name': 'addition', 'arguments': '{"a": "4", "b": "6"}'}, 'type': 'function'}]
```

### 4. 将执行工具调用的结果添加到消息历史记录并继续对话

```python
if function_call:
    # append the assistant tool call msg
    messages.append({"role": "assistant", "tool_calls": function_call})
    # append the result of the tool call in openai format, in this case, the value of add 6 to 4 is 10.
    messages.append({'role': 'tool', 'tool_call_id': function_call[0]["id"], 'name': function_call[0]["function"]["name"], 'content': '10'})
    raw_output1 = run_model(messages, functions)
    # Check if there's a function call

    function_call = postprocess_output(raw_output1)
    if function_call:
        print(function_call)
    else:
        print(raw_output)
```

LLM将再次进行调用
```
[{'id': '2ffc3de4', 'function': {'name': 'addition', 'arguments': '{"a": "10", "b": "2"}'}, 'type': 'function'}]
```

## 框架版本
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1

## 限制和偏见

虽然该模型在广泛的任务上表现良好，但它可能仍然会产生偏见或错误的输出。用户在使用该模型时应谨慎，并在敏感或高风险应用中保持批判性的判断。模型的输出受其训练数据的影响，可能包含固有的偏见。

## 道德考虑

用户应确保该模型的部署符合道德指南，并考虑所生成文本的潜在社会影响。我们强烈反对将该模型滥用于生成有害或误导性内容。

## 致谢

我们感谢Alibaba Cloud提供的基础模型。

## 联系方式

如果对此模型有任何问题或意见或建议，请联系 [rubra团队](mailto:rubra@acorn.io)。

## 引用信息

如果您使用了这项工作，请引用如下：

```
@misc {rubra_ai_2024,
	author       = { Sanjay Nadhavajhala and Yingbei Tong },
	title        = { Rubra-Qwen2-7B-Instruct },
	year         = 2024,
	url          = { https://huggingface.co/rubra-ai/Qwen2-7B-Instruct },
	doi          = { 10.57967/hf/2658 },
	publisher    = { Hugging Face }
}
```