---
title: starcoder2-15b-instruct-GPTQ
canonical_url: "https://www.modelscope.cn/models/TechxGenus-MS/starcoder2-15b-instruct-GPTQ"
md_url: "https://www.modelscope.cn/models/TechxGenus-MS/starcoder2-15b-instruct-GPTQ.md"
repository: TechxGenus-MS/starcoder2-15b-instruct-GPTQ
last_updated: 2024-12-11
license: bigcode-openrail-m
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - starcoder2
architectures:
  - Starcoder2ForCausalLM
parameters: 2.7B
tensor_type:
  - I32
  - F16
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "vllm 0.9.2"
downloads: 1249
stars: 0
tags:
  - code
  - starcoder2
---

# starcoder2-15b-instruct-GPTQ

> starcoder2-15b-instruct-GPTQ - TechxGenus-MS 在 ModelScope 开源的模型。GPTQ quantized version of starcoder2-15b-instruct model.

TechxGenus-MS/starcoder2-15b-instruct-GPTQ 是 ModelScope 魔搭社区上的 2.7B 参数text-generation模型，采用 bigcode-openrail-m 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、vllm 0.9.2 部署。

- **Repository**: TechxGenus-MS/starcoder2-15b-instruct-GPTQ
- **License**: bigcode-openrail-m
- **Tasks**: text-generation
- **Parameters**: 2.7B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, vllm 0.9.2
- **Tags**: code, starcoder2
- **Downloads**: 1249
- **Stars**: 0
- **Last updated**: 2024-12-11

Source: https://www.modelscope.cn/models/TechxGenus-MS/starcoder2-15b-instruct-GPTQ

---

<p align="center">
<img width="300px" alt="starcoder2-instruct" src="https://huggingface.co/TechxGenus/starcoder2-15b-instruct/resolve/main/starcoder2-instruct.jpg">
</p>

GPTQ quantized version of starcoder2-15b-instruct model.

---

### starcoder2-instruct

We've fine-tuned starcoder2-15b with an additional 0.7 billion high-quality, code-related tokens for 3 epochs. We used DeepSpeed ZeRO 3 and Flash Attention 2 to accelerate the training process. It achieves **77.4 pass@1** on HumanEval-Python. This model operates using the Alpaca instruction format (excluding the system prompt).

### Usage

Here give some examples of how to use our model:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
PROMPT = """### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
tokenizer = AutoTokenizer.from_pretrained("TechxGenus/starcoder2-15b-instruct")
model = AutoModelForCausalLM.from_pretrained(
    "TechxGenus/starcoder2-15b-instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=2048)
print(tokenizer.decode(outputs[0]))
```

With text-generation pipeline:


```python
from transformers import pipeline
import torch
PROMPT = """### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
generator = pipeline(
    model="TechxGenus/starcoder2-15b-instruct",
    task="text-generation",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
result = generator(prompt, max_length=2048)
print(result[0]["generated_text"])
```

### Note

Model may sometimes make errors, produce misleading contents, or struggle to manage tasks that are not related to coding. It has undergone very limited testing. Additional safety testing should be performed before any real-world deployments.