---
title: sd3-long-captioner-v2
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/sd3-long-captioner-v2"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/sd3-long-captioner-v2.md"
repository: AI-ModelScope/sd3-long-captioner-v2
last_updated: 2024-09-18
license: apache-2.0
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
model_type:
  - paligemma
architectures:
  - PaliGemmaForConditionalGeneration
parameters: 2.9B
tensor_type:
  - F32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
language:
  - en
inference_backends:
  - "deploy_task vlm/text/emb"
  - "vllm 0.9.2"
downloads: 2299
stars: 0
tags:
  - art
---

# sd3-long-captioner-v2

> sd3-long-captioner-v2 - AI-ModelScope 在 ModelScope 开源的模型。Fine-tuned version of PaliGemma 224x224 on google/docci and google/imageinwords datasets.

AI-ModelScope/sd3-long-captioner-v2 是 ModelScope 魔搭社区上的 2.9B 参数image-text-to-text模型，采用 apache-2.0 许可，可用 deploy_task vlm/text/emb、vllm 0.9.2 部署。

- **Repository**: AI-ModelScope/sd3-long-captioner-v2
- **License**: apache-2.0
- **Tasks**: image-text-to-text
- **Parameters**: 2.9B
- **Inference backends**: deploy_task vlm/text/emb, vllm 0.9.2
- **Tags**: art
- **Downloads**: 2299
- **Stars**: 0
- **Last updated**: 2024-09-18

Source: https://www.modelscope.cn/models/AI-ModelScope/sd3-long-captioner-v2

---

Fine-tuned version of PaliGemma 224x224 on [google/docci](https://huggingface.co/datasets/google/docci) and [google/imageinwords](https://huggingface.co/datasets/google/imageinwords) datasets.
```
pip install git+https://github.com/huggingface/transformers
```

```python
from transformers import AutoProcessor, PaliGemmaForConditionalGeneration
from PIL import Image
import requests
import torch

model_id = "gokaygokay/sd3-long-captioner-v2"

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true"
image = Image.open(requests.get(url, stream=True).raw)

model = PaliGemmaForConditionalGeneration.from_pretrained(model_id).eval()
processor = AutoProcessor.from_pretrained(model_id)

## prefix
prompt = "caption en"
model_inputs = processor(text=prompt, images=image, return_tensors="pt")
input_len = model_inputs["input_ids"].shape[-1]

with torch.inference_mode():
    generation = model.generate(**model_inputs, repetition_penalty=1.10, max_new_tokens=256, do_sample=False)
    generation = generation[0][input_len:]
    decoded = processor.decode(generation, skip_special_tokens=True)
    print(decoded)
 
```