---
title: internlm-xcomposer2-vl-7b-4bit
canonical_url: "https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b-4bit"
md_url: "https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b-4bit.md"
repository: Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b-4bit
chinese_name: "浦语·灵笔2-视觉问答-7B-4bit"
last_updated: 2025-01-15
license: other
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - internlm
architectures:
  - InternLMXComposer2ForCausalLM
parameters: 2.6B
tensor_type:
  - F16
  - I32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - Pytorch
inference_backends:
  - "deploy_task vlm"
  - "lmdeploy_turbomind 0.9.1"
downloads: 1456
stars: 3
---

# internlm-xcomposer2-vl-7b-4bit

> internlm-xcomposer2-vl-7b-4bit - Shanghai_AI_Laboratory 在 ModelScope 开源的模型。InternLM-XComposer2 is a vision-language large model (VLLM) based on InternLM2 for advanced text-image comprehension and composition.

Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b-4bit 是 ModelScope 魔搭社区上的 2.6B 参数text-generation模型，采用 other 许可，可用 deploy_task vlm、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b-4bit
- **License**: other
- **Tasks**: text-generation
- **Parameters**: 2.6B
- **Inference backends**: deploy_task vlm, lmdeploy_turbomind 0.9.1
- **Downloads**: 1456
- **Stars**: 3
- **Last updated**: 2025-01-15

Source: https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b-4bit

---

<p align="center">
    <img src="logo_en.png" width="400"/>
<p>

<p align="center">
    <b><font size="6">InternLM-XComposer2</font></b> 
<p>

<div align="center">

[💻Github Repo](https://github.com/InternLM/InternLM-XComposer)

[Paper](https://arxiv.org/abs/2401.16420)

</div>

**InternLM-XComposer2** is a vision-language large model (VLLM) based on [InternLM2](https://github.com/InternLM/InternLM) for advanced text-image comprehension and composition. 

We release InternLM-XComposer2 series in two versions:

- InternLM-XComposer2-VL: The pretrained VLLM model with InternLM2 as the initialization of the LLM, achieving strong performance on various multimodal benchmarks.
- InternLM-XComposer2: The finetuned VLLM for *Free-from Interleaved Text-Image Composition*.

This is the 4-bit version of InternLM-XComposer2-VL, install the latest version of [auto_gptq](https://github.com/AutoGPTQ/AutoGPTQ#quick-installation) before using.

## Quickstart
We provide a simple example to show how to use InternLM-XComposer with 🤗 Transformers.
```python
import torch, auto_gptq
from transformers import AutoModel, AutoTokenizer 
from auto_gptq.modeling import BaseGPTQForCausalLM

auto_gptq.modeling._base.SUPPORTED_MODELS = ["internlm"]
torch.set_grad_enabled(False)

class InternLMXComposer2QForCausalLM(BaseGPTQForCausalLM):
    layers_block_name = "model.layers"
    outside_layer_modules = [
        'vit', 'vision_proj', 'model.tok_embeddings', 'model.norm', 'output', 
    ]
    inside_layer_modules = [
        ["attention.wqkv.linear"],
        ["attention.wo.linear"],
        ["feed_forward.w1.linear", "feed_forward.w3.linear"],
        ["feed_forward.w2.linear"],
    ]
 
# init model and tokenizer
model = InternLMXComposer2QForCausalLM.from_quantized(
  'internlm/internlm-xcomposer2-vl-7b-4bit', trust_remote_code=True, device="cuda:0").eval()
tokenizer = AutoTokenizer.from_pretrained(
  'internlm/internlm-xcomposer2-vl-7b-4bit', trust_remote_code=True)

text = '<ImageHere>Please describe this image in detail.'
image = 'examples/image1.webp'
with torch.cuda.amp.autocast(): 
  response, _ = model.chat(tokenizer, query=query, image=image, history=[], do_sample=False) 
print(response)
#The image features a quote by Oscar Wilde, "Live life with no excuses, travel with no regrets." 
#The quote is displayed in white text against a dark background. In the foreground, there are two silhouettes of people standing on a hill at sunset. 
#They appear to be hiking or climbing, as one of them is holding a walking stick. 
#The sky behind them is painted with hues of orange and purple, creating a beautiful contrast with the dark figures.

```


### Open Source License
The code is licensed under Apache-2.0, while model weights are fully open for academic research and also allow free commercial usage. To apply for a commercial license, please fill in the application form (English)/申请表（中文）. For other questions or collaborations, please contact internlm@pjlab.org.cn.