---
title: SeaLLM-7B-chat
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/SeaLLM-7B-chat"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/SeaLLM-7B-chat.md"
repository: AI-ModelScope/SeaLLM-7B-chat
last_updated: 2023-12-14
license: other
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - llama
architectures:
  - LlamaForCausalLM
parameters: 6.9B
tensor_type:
  - BF16
library_name:
  - pytorch
  - safetensors
frameworks:
  - Pytorch
language:
  - en
  - vi
  - id
  - ms
  - th
  - km
  - lo
  - my
  - tl
  - zh
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 78
stars: 4
---

# SeaLLM-7B-chat

> SeaLLM-7B-chat - AI-ModelScope 在 ModelScope 开源的模型。SeaLLMs - Large Language Models for Southeast Asia

AI-ModelScope/SeaLLM-7B-chat 是 ModelScope 魔搭社区上的 6.9B 参数text-generation模型，采用 other 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: AI-ModelScope/SeaLLM-7B-chat
- **License**: other
- **Tasks**: text-generation
- **Parameters**: 6.9B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 78
- **Stars**: 4
- **Last updated**: 2023-12-14

Source: https://www.modelscope.cn/models/AI-ModelScope/SeaLLM-7B-chat

---

<p align="center">
  <img src="seal_logo.png" width="200" />
</p>

# SeaLLMs - Large Language Models for Southeast Asia


<p align="center">
<a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a>
&nbsp;&nbsp;
<a href="https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 DEMO</a>
&nbsp;&nbsp;
<a href="https://github.com/DAMO-NLP-SG/SeaLLMs" target="_blank" rel="noopener">Github</a>
&nbsp;&nbsp;
<a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a>
</p>

## SeaLLM-chat-7B

This a **7B Chat** version of SeaLLMs. It Vietnamese 🇻🇳, Indonesian 🇮🇩, Thai 🇹🇭, Malay 🇲🇾, Khmer 🇰🇭, Lao 🇱🇦, Tagalog 🇵🇭 and Burmese 🇲🇲. 
It may have lower capability than the 13B models but it is much more memory-efficient and faster.

Visit our <a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a> and <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a> for more details.

<blockquote style="color:red">
<p><strong style="color: red">Terms of Use and License</strong>: 
By using our released weights, codes, and demos, you agree to and comply with the terms and conditions specified in our <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b/edit/main/LICENSE" target="_blank" rel="noopener">SeaLLMs Terms Of Use</a>.
</blockquote>

> **Disclaimer**:
> We must note that even though the weights, codes, and demos are released in an open manner, similar to other pre-trained language models, and despite our best efforts in red teaming and safety fine-tuning and enforcement, our models come with potential risks, including but not limited to inaccurate, misleading or potentially harmful generation.
> Developers and stakeholders should perform their own red teaming and provide related security measures before deployment, and they must abide by and comply with local governance and regulations.
> In no event shall the authors be held liable for any claim, damages, or other liability arising from the use of the released weights, codes, or demos.

> The logo was generated by DALL-E 3.

## 示例代码:

SeaLLM models work the same way as Llama-2, so the Llama-2 generation codebase should be sufficient to run. 
However, as this is a chat model, you should wrap the prompt/instruction using the following format function.

You should also turn off add_special_tokens with `tokenizer.add_special_tokens = False`.

```python
from modelscope import AutoTokenizer,Model
from modelscope import snapshot_download
import torch 
from typing import List, Tuple

BOS_TOKEN = '<s>'
EOS_TOKEN = '</s>'

B_INST, E_INST = "[INST]", "[/INST]"
B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"

SYSTEM_PROMPT = """You are a multilingual, helpful, respectful and honest assistant. \
Please always answer as helpfully as possible, while being safe. Your \
answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure \
that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not \
correct. If you don't know the answer to a question, please don't share false information.

As a multilingual assistant, you must respond and follow instructions in the native language of the user by default, unless told otherwise. \
Your response should adapt to the norms and customs of the respective language and culture.
"""

def chat_multiturn_seq_format(
    message: str,
    history: List[Tuple[str, str]] = [], 
):
    """
    ```
        <bos>[INST] B_SYS SytemPrompt E_SYS Prompt [/INST] Answer <eos>
        <bos>[INST] Prompt [/INST] Answer <eos>
        <bos>[INST] Prompt [/INST]
    ```
    As the format auto-add <bos>, please turn off add_special_tokens with `tokenizer.add_special_tokens = False`
    Inputs:
      message: the current prompt
      history: list of list indicating previous conversation. [[message1, response1], [message2, response2]]
    Outputs:
      full_prompt: the prompt that should go into the chat model

    e.g:
      full_prompt = chat_multiturn_seq_format("Hello world")
      output = model.generate(tokenizer.encode(full_prompt, add_special_tokens=False), ...)
    """
    text = ''
    for i, (prompt, res) in enumerate(history):
        if i == 0:
            text += f"{BOS_TOKEN}{B_INST} {B_SYS} {SYSTEM_PROMPT} {E_SYS} {prompt} {E_INST}"
        else:
            text += f"{BOS_TOKEN}{B_INST} {prompt}{end_instr}"
        if res is not None:
            text += f" {res} {EOS_TOKEN} "
    if len(history) == 0 or text.strip() == '':
        text = f"{BOS_TOKEN}{B_INST} {B_SYS} {SYSTEM_PROMPT} {E_SYS} {message} {E_INST}"
    else:
        text += f"{BOS_TOKEN}{B_INST} {message} {E_INST}"
    return text

local_dir = snapshot_download("AI-ModelScope/SeaLLM-7B-chat",revision='master')

model = Model.from_pretrained(local_dir, revision='master', device_map='auto', torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(local_dir, revision='master')

full_prompt = chat_multiturn_seq_format("Hello world")
inputs = tokenizer(full_prompt, add_special_tokens=False, return_tensors="pt")
# Generate
generate_ids = model.generate(inputs.input_ids.to(model.device),  max_length=512,do_sample=True,top_k=10,num_return_sequences=1)
print(tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0])
```


## Citation

If you find our project useful, we hope you would kindly star our repo and cite our work as follows: Corresponding Author: [l.bing@alibaba-inc.com](mailto:l.bing@alibaba-inc.com)

```
@article{damonlpsg2023seallm,
  author = {Xuan-Phi Nguyen*, Wenxuan Zhang*, Xin Li*, Mahani Aljunied*,
            Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang,
            Chaoqun Liu, Hang Zhang, Lidong Bing},
  title = {SeaLLMs - Large Language Models for Southeast Asia},
  year = 2023,
  Eprint = {arXiv:2312.00738},
}
```