---
title: falcon-7b-code-alpaca-lora
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/falcon-7b-code-alpaca-lora"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/falcon-7b-code-alpaca-lora.md"
repository: AI-ModelScope/falcon-7b-code-alpaca-lora
last_updated: 2023-10-30
license: cc-by-nc-4.0
pipeline_tag: text-generation
tasks:
  - text-generation
library_name:
  - pytorch
  - transformer
frameworks:
  - pytorch
language:
  - en
downloads: 26
stars: 0
tags:
  - text-generation
---

# falcon-7b-code-alpaca-lora

> falcon-7b-code-alpaca-lora - AI-ModelScope 在 ModelScope 开源的模型。LLM Generation models trained by Jina AI, Finetuner team.

AI-ModelScope/falcon-7b-code-alpaca-lora 是 ModelScope 魔搭社区上的text-generation模型，采用 cc-by-nc-4.0 许可。

- **Repository**: AI-ModelScope/falcon-7b-code-alpaca-lora
- **License**: cc-by-nc-4.0
- **Tasks**: text-generation
- **Tags**: text-generation
- **Downloads**: 26
- **Stars**: 0
- **Last updated**: 2023-10-30

Source: https://www.modelscope.cn/models/AI-ModelScope/falcon-7b-code-alpaca-lora

---

<br><br>

<p align="center">
<img src="https://github.com/jina-ai/finetuner/blob/main/docs/_static/finetuner-logo-ani.svg?raw=true" alt="Finetuner logo: Finetuner helps you to create experiments in order to improve embeddings on search tasks. It accompanies you to deliver the last mile of performance-tuning for neural search applications." width="150px">
</p>


<p align="center">
<b>LLM Generation models trained by Jina AI, Finetuner team.</b>

This repo contains the lora weights (8bit) for Falcon-7b
fit on the [Code Alpaca](https://huggingface.co/datasets/sahil2801/CodeAlpaca-20k) dataset.

## Reproduction
This version of the weights was trained with the following hyperparameters:

- Epochs: 6 
- Batch size: 128
- Micro batch size: 8
- Learning rate: 3e-4
- Lora _r_: 8
- Lora target modules: query_key_value

You can reproduce using this repository: 

https://github.com/jina-ai/jerboa

Make sure you install requirements and finetune using this command using the following command:

```
python finetune.py \
--base-model tiiuae/falcon-7b --lora-target-modules query_key_value \
--data-path sahil2801/CodeAlpaca-20k --output-dir ./lora-alpaca-code \
--batch-size 128 --micro-batch-size 8 --eval-limit 45 \
--eval-file code_eval.jsonl --wandb-project jerboa --wandb-log-model \
 --wandb-watch gradients --num-epochs 6
```

## Inference

```Python
import torch
from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForCausalLM


TOKENIZER_SOURCE = 'tiiuae/falcon-7b'
BASE_MODEL = 'tiiuae/falcon-7b'
LORA_REPO = 'jinaai/falcon-7b-code-alpaca-lora'
DEVICE = "cuda"

PROMPT = """
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Write a for loop in python

### Input:

### Response:
"""
model = AutoModelForCausalLM.from_pretrained(
    pretrained_model_name_or_path=BASE_MODEL,
    torch_dtype=torch.float16,
    trust_remote_code=True,
    device_map='auto',
)

model = PeftModel.from_pretrained(
    model=model,
    model_id=LORA_REPO,
)
model.eval()

tokenizer = AutoTokenizer.from_pretrained(
    TOKENIZER_SOURCE,
    trust_remote_code=True,
    padding_side='left',
)
tokenizer.pad_token = tokenizer.eos_token

inputs = tokenizer(PROMPT, return_tensors="pt")
input_ids = inputs["input_ids"].to(DEVICE)
input_attention_mask = inputs["attention_mask"].to(DEVICE)

with torch.no_grad():
    generation_output = model.generate(
        input_ids=input_ids,
        attention_mask=input_attention_mask,
        return_dict_in_generate=True,
        max_new_tokens=32,
        eos_token_id=tokenizer.eos_token_id,
    )
generation_output = generation_output.sequences[0]
output = tokenizer.decode(generation_output, skip_special_tokens=True)

print(output)

```

## Contact

Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.