---
title: RSVLM_SFT
canonical_url: "https://www.modelscope.cn/datasets/KAIWANG/RSVLM_SFT"
md_url: "https://www.modelscope.cn/datasets/KAIWANG/RSVLM_SFT.md"
repository: KAIWANG/RSVLM_SFT
chinese_name: RSVLM_SFT
last_updated: 2026-09-16
license: apache-2.0
storage_size: "16 GB"
downloads: 8261
stars: 0
---

# RSVLM_SFT

> RSVLM_SFT - KAIWANG 在 ModelScope 开源的数据集。Mirror of RL-MIND/RSVLM_SFT; paper links and citations in README.

KAIWANG/RSVLM_SFT 是 ModelScope 魔搭社区上的数据集，存储大小 16 GB，采用 apache-2.0 许可。

- **Repository**: KAIWANG/RSVLM_SFT
- **License**: apache-2.0
- **Storage size**: 16 GB
- **Downloads**: 8261
- **Stars**: 0
- **Last updated**: 2026-09-16

Source: https://www.modelscope.cn/datasets/KAIWANG/RSVLM_SFT

---

<table align="center" role="presentation" style="margin:0 auto; border:0; background:transparent;">
  <tr style="border:0; background:transparent;">
    <td align="center" style="border:0; background:transparent; padding:0 8px;">
      <img src="./assets/rl-mind-logo-v2.webp" alt="RL-MIND research group logo" width="160" height="160" loading="eager" fetchpriority="high" decoding="async" style="width:160px; height:160px; object-fit:contain;" />
    </td>
    <td align="center" style="border:0; background:transparent; padding:0 8px;">
      <img src="./assets/dataset-logo-v2.webp" alt="RSVLM_SFT dataset logo" width="160" height="160" loading="eager" fetchpriority="high" decoding="async" style="width:160px; height:160px; object-fit:contain;" />
    </td>
  </tr>
</table>

<h1 align="center">RSVLM_SFT</h1>
<p align="center"><strong>Remote-Sensing Data for Vision-Language Instruction Tuning</strong></p>

<p align="center">
  <a href="https://arxiv.org/abs/2512.24022"><img src="https://img.shields.io/badge/arXiv-2512.24022-B31B1B.svg" alt="Paper on arXiv" /></a>
  <a href="https://huggingface.co/datasets/RL-MIND/RSVLM_SFT"><img src="https://img.shields.io/badge/🤗_Hugging_Face-Dataset-FFD21E" alt="Dataset on Hugging Face" /></a>
  <a href="https://modelscope.cn/datasets/KAIWANG/RSVLM_SFT"><img src="https://img.shields.io/badge/ModelScope-Dataset-624AFF" alt="Dataset on ModelScope" /></a>
</p>
<p align="center"><a href="https://yunkaidang.github.io/bibliography/dang2025fuse-rsvlm/">Project</a> · <a href="https://arxiv.org/abs/2512.24022">Paper</a> · <a href="https://github.com/Yunkaidang/RSVLM">Code</a></p>
<p align="center"><strong>English</strong> | <a href="./README_ZH.md">中文</a></p>

## 📚 Introduction

**RSVLM_SFT** is the remote-sensing data repository associated with supervised instruction tuning for **MF-RSVLM**, the model presented in **FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensing**. It brings together image resources and source-specific annotations used in remote-sensing vision-language research.

This repository keeps the original dataset directory structure. The accompanying project provides model training, inference, and evaluation code.

[At a glance](#at-a-glance) · [Data format](#data-format) · [Quick start](#quick-start) · [Model workflow](#model-workflow) · [License and sources](#license-and-sources) · [Citation](#citation)

<a id="at-a-glance"></a>
## 📊 At a Glance

| Item | Description |
|---|---|
| Domain | Remote sensing |
| Associated training stage | Supervised fine-tuning / instruction tuning |
| Associated method | MF-RSVLM / FUSE-RSVLM |
| Released files | JPG, PNG and TIF images; JSON annotations; TAR archives |
| Organization | Source-specific directories and formats |
| Hugging Face access | Automatic access approval; sign in and accept the access conditions on the dataset page |
| Repository license | Apache-2.0 |

The source card does not specify a verified total of SFT conversations. File counts and archive members should not be treated as training-example counts. The **1.4M VersaD image-text pairs** described in the model workflow refer to its separate pretraining stage.

<a id="data-format"></a>
## 📦 Data Organization and Format

Top-level data directories include:

| Group | Directory names as released |
|---|---|
| A–M | `BANDON`, `DOIR-RSVG`, `DOTA-train`, `FBP`, `METER-ML`, `MSAR`, `Mts-WH` |
| N–V | `NWPU-RESISC45`, `RSITMD`, `RSVQA_LR-comp`, `RSVQA_LR-presence`, `RSVQA_LR-rural_urban`, `UCM`, `VRSBench` |
| Other released names | `corwdAI`, `crowdAI`, `deepglobe_lc`, `dior` |

Directory spelling and capitalization are preserved. In particular, `corwdAI` and `crowdAI` are distinct released paths.

Formats vary by source. For example, JSON files under `VRSBench/train/annotations/Annotations_train/` contain the following top-level fields:

| Field | Contents |
|---|---|
| `image` | Referenced image filename. |
| `caption` | Image caption. |
| `objects` | Object-level descriptions and geometry. |
| `qa_pairs` | Questions and answers with IDs and question types. |

These fields describe the inspected VRSBench annotation format, not a shared schema for all directories. Preserve original paths, annotation structures and source split directories when preparing training data.

<a id="quick-start"></a>
## 🚀 Quick Start

Install the ModelScope SDK, then download files selected by path.

This example downloads only VRSBench training annotations; download the corresponding images separately for training. Adjust `allow_patterns` to select other source files, or remove it to download the full repository.

```python
from modelscope.hub.snapshot_download import dataset_snapshot_download

root = dataset_snapshot_download(
    dataset_id="KAIWANG/RSVLM_SFT",
    revision="master",
    local_dir="./RSVLM_SFT",
    allow_patterns=["VRSBench/train/annotations/Annotations_train/*.json"],
)
```

Download the required source files before configuring the project training scripts. This repository is a collection of files and archives; the card does not define one universal `load_dataset(..., split="train")` schema for every source.

<a id="model-workflow"></a>
## 🛠️ Associated Model Workflow

The following instructions retain the accompanying MF-RSVLM project's workflow. Run these commands in the **code repository**, after preparing data paths and model checkpoints. They do not execute from the dataset repository alone.

<details>
<summary>Model context, installation, training, inference and evaluation</summary>

### Model Context

MF-RSVLM combines a CLIP ViT-L/14 336px visual encoder, a two-layer MLP projector, and Vicuna-7B v1.5. Training consists of modality-alignment pretraining on VersaD (1.4M image-text pairs), followed by supervised instruction tuning.

### Installation

```bash
git clone https://github.com/Yunkaidang/RSVLM.git MF-RSVLM
cd MF-RSVLM
conda create -n mf-rsvlm
conda activate mf-rsvlm
pip install -r requirements.txt
```

The code repository contains `mfrsvlm/` (model, training, conversation and utilities), `scripts/` (inference, evaluation, data preparation and ZeRO configurations), and `requirements.txt`. Prepare checkpoints under `checkpoints/` and component models under `models/` as required by the project scripts.

### Model and Pretraining Resources

| Resource | Link |
|---|---|
| MF-RSVLM pretraining checkpoint | [mf_rsvlm_7b_pretrained](https://huggingface.co/FelixKAI/mf_rsvlm_7b_pretrained) |
| MF-RSVLM SFT checkpoint | [mfrsvlm-7b_sft](https://huggingface.co/FelixKAI/mfrsvlm-7b_sft) |
| CLIP visual encoder | [clip-vit-large-patch14-336](https://huggingface.co/openai/clip-vit-large-patch14-336) |
| Vicuna language model | [vicuna-7b-v1.5](https://huggingface.co/lmsys/vicuna-7b-v1.5) |
| LLaVA-1.5 MLP projector | [llava-v1.5-mlp2x-336px-pretrain-vicuna-7b-v1.5](https://huggingface.co/liuhaotian/llava-v1.5-mlp2x-336px-pretrain-vicuna-7b-v1.5/tree/main) |
| Separate pretraining dataset | [VHM_VersaD](https://huggingface.co/datasets/FitzPC/VHM_VersaD) |

### Training

```bash
# Modality-alignment pretraining
sh scripts/rs/slurm_pretrain.sh

# Supervised instruction tuning
sh scripts/rs/slurm_finetune.sh
```

### Single-Sample Inference

```bash
CUDA_VISIBLE_DEVICES=0 python scripts/run_mfrsvlm_inference.py \
  --model-path checkpoints/mfrsvlm-7b_sft \
  --image-path /path/to/image.png \
  --prompt "What is shown in the image?"
```

### Web Demo

```bash
CUDA_VISIBLE_DEVICES=0 python scripts/run_mf-rsvlm_web_server.py \
  --model-path checkpoints/mfrsvlm-7b_sft \
  --host 0.0.0.0 \
  --port 7860
```

Open `http://localhost:7860`, upload an image, and enter a question. See the [project's demo screenshot](https://github.com/Yunkaidang/RSVLM/blob/main/asserts/result.png).

### Evaluation

```bash
git clone https://github.com/fitzpchao/RSEvalKit
cd RSEvalKit
conda create -n rseval
conda activate rseval
pip install -r requirements.txt
```

Prepare the required model weights and datasets, then follow the [RSEvalKit documentation](https://github.com/fitzpchao/RSEvalKit).

</details>

<a id="license-and-sources"></a>
## 📜 License, Sources and Acknowledgements

The repository retains its **Apache-2.0** license declaration. Source datasets and pretrained components retain their own terms and attribution; the repository declaration does not replace those terms.

Dataset repositories: [Hugging Face source](https://huggingface.co/datasets/RL-MIND/RSVLM_SFT) · [ModelScope mirror](https://modelscope.cn/datasets/KAIWANG/RSVLM_SFT). The source data directory names are listed above, and their file organization is preserved.

The accompanying project acknowledges [Vicuna](https://github.com/lm-sys/FastChat#vicuna-weights), [LLaVA](https://github.com/haotian-liu/LLaVA), [ShareGPT4V](https://github.com/InternLM/InternLM-XComposer/tree/main/projects/ShareGPT4V), [LLaMA](https://github.com/facebookresearch/llama), and [VHM](https://github.com/opendatalab/VHM).

<a id="citation"></a>
## 📖 Citation

```bibtex
@article{dang2025fuse,
  title={FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensing},
  author={Dang, Yunkai and Wang, Donghao and Yang, Jiacheng and Jiang, Yifan and Zhu, Meiyi and Yang, Yuekun and Wang, Cong and Fan, Qi and Li, Wenbin and Gao, Yang},
  journal={arXiv preprint arXiv:2512.24022},
  year={2025}
}
```