--- title: DeltaV-2B canonical_url: "https://www.modelscope.cn/models/wpj2003/DeltaV-2B" md_url: "https://www.modelscope.cn/models/wpj2003/DeltaV-2B.md" repository: wpj2003/DeltaV-2B last_updated: 2026-07-10 license: "Apache License 2.0" pipeline_tag: any-to-any tasks: - any-to-any model_type: - qwen3_vl architectures: - Qwen3VLModel base_model_relation: finetune parameters: 2.6B tensor_type: - BF16 library_name: - safetensors - pytorch frameworks: - Pytorch downloads: 14 stars: 0 --- # DeltaV-2B > DeltaV-2B - wpj2003 在 ModelScope 开源的模型。当前模型的贡献者未提供更加详细的模型介绍。模型文件和权重，可浏览“模型文件”页面获取。您可以通过如下git clone命令，或者ModelScope SDK来下载模型 wpj2003/DeltaV-2B 是 ModelScope 魔搭社区上的 2.6B 参数any-to-any模型，采用 Apache License 2.0 许可。 - **Repository**: wpj2003/DeltaV-2B - **License**: Apache License 2.0 - **Tasks**: any-to-any - **Parameters**: 2.6B - **Downloads**: 14 - **Stars**: 0 - **Last updated**: 2026-07-10 Source: https://www.modelscope.cn/models/wpj2003/DeltaV-2B --- ### 当前模型的贡献者未提供更加详细的模型介绍。模型文件和权重，可浏览“模型文件”页面获取。 #### 您可以通过如下git clone命令，或者ModelScope SDK来下载模型 SDK下载 ```bash #安装ModelScope pip install modelscope ``` ```python #SDK模型下载 from modelscope import snapshot_download model_dir = snapshot_download('wpj2003/ViMo') ``` Git下载 ``` #Git模型下载 git clone https://www.modelscope.cn/wpj2003/ViMo.git ```

# DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models **A unified large multimodal model that thinks with visual state updates—modeling only the sparse, reasoning-critical changes across reasoning steps instead of regenerating full images.** [![arXiv](https://img.shields.io/badge/Arxiv-DeltaV-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2607.08434) [![HuggingFace](https://img.shields.io/badge/HuggingFace-DeltaV--2B-yellow.svg?logo=HuggingFace)](https://huggingface.co/wpj20000/DeltaV-2B/tree/main) [![ModelScope](https://img.shields.io/badge/ModelScope-DeltaV--2B-green.svg)](https://www.modelscope.cn/models/wpj2003/DeltaV-2B) [![Dataset](https://img.shields.io/badge/Dataset-StructCoT-orange.svg)](https://www.modelscope.cn/datasets/wpj2003/StructCoT) [![Website](https://img.shields.io/badge/Website-DeltaV-blue.svg)](https://pengjie-w.github.io/DeltaV/) [![Demo](https://img.shields.io/badge/Demo-Live-purple.svg)](http://vlrlabmonkey.xyz:10088/)

--- ## News * ```2026.07.09 ``` 🚀 We release [DeltaV-2B](https://huggingface.co/wpj20000/DeltaV-2B/tree/main), a unified large multimodal model for interleaved multimodal reasoning. ## Introduction DeltaV is a unified large multimodal model (ULMM) designed to think with visual state updates during interleaved multimodal reasoning. Conditioned on historical visual states, it incrementally predicts compact **visual update tokens** that capture sparse but reasoning-critical changes across reasoning steps, avoiding repeated modeling of unchanged content. Token budgets are dynamically allocated by the **TSIM Router** according to temporal visual variation, and visual states are encoded by the **TSIM-Tok** tokenizer. This repository releases the **DeltaV-2B ULMM** together with the **TSIM-Tok tokenizer**, inference scripts, and tiny samples. ## TODO / Roadmap This release focuses on **inference** with the pretrained DeltaV-2B checkpoint. The following components are not included yet and will be released in a future update: - [ ] **StructCoT dataset** — the full StructCoT dataset and training data. (A small inference example, `data/struct_infer_sample.json`, is included so inference is runnable; the benchmark numbers in the tables below are kept as a record.) - [ ] **Zebra-CoT / StructCoT evaluation (scoring) tools** — the LLM-API-based scorers and their guide, pending the public StructCoT release. - [ ] **DeltaV training** — the two-stage DeltaV ULMM training scripts and configs. - [ ] **TSIM-Tok training and testing** — the TSIM-Tok tokenizer training and reconstruction-evaluation code (the tokenizer *model* is retained as DeltaV's visual backbone). ## DeltaV Workflow https://github.com/user-attachments/assets/724b2ff7-279a-4139-a9a0-de734014431d ## Repository Layout ```text deltav/ DeltaV model code: modeling, processing, configuration, backbone tsim_tok/ TSIM-Tok visual tokenizer and TSIM Router inference/ Inference scripts/ Ready-to-run scripts for DeltaV inference and data utilities configs/ Model and acceleration configs data/ Tiny samples docs/ Extended tutorials and README media assets tools/ Data processing and inference post-processing ``` ## Installation See [INSTALL.md](INSTALL.md) for the full setup guide. Quick version: ```bash conda create -n deltav python=3.10 -y && conda activate deltav pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121 pip install -r requirements.txt ``` ### Download Checkpoints Download our models from Huggingface. ```bash pip install huggingface_hub python tools/download_model.py -n DeltaV-2B # DeltaV-2B (legacy CLI identifier) ``` You can also download our models from ModelScope. ```bash pip install modelscope python tools/download_model.py -t modelscope -n DeltaV-2B # DeltaV-2B (legacy CLI identifier) ``` The released checkpoint is placed under `weights/`: ```text weights/ deltav_2b/ ``` ## Inference > **DeltaV training is not included in this release.** The two-stage DeltaV ULMM training recipe (on top of a frozen TSIM-Tok) will be added back later — see [TODO / Roadmap](#todo--roadmap). ### DeltaV Inference ```bash # Pure inference for evaluation. Outputs text-only .json, then merge + extract answers. MODEL_PATH=weights/deltav_2b \ JSON_PATH=data/zebra_infer_sample.json \ bash scripts/deltav/infer_deltav.sh ``` To also decode and save generated images, set `VIS_ARGS`. This streams results to `.jsonl` and skips merge/extract steps. Add `--concat_gt_images` to dump a ground-truth montage alongside each prediction. ```bash MODEL_PATH=weights/deltav_2b \ JSON_PATH=data/zebra_infer_sample.json \ VIS_ARGS="--decode_and_save_image --concat_gt_images" \ bash scripts/deltav/infer_deltav.sh ``` ### Input format & token budgets Each inference sample carries the prompt, its input/output image paths, and (optionally) the per-image incremental token budget: ```json { "config": "Visual Logic & Strategic Games - Tetris", "input_prompt": "Fill the entire grid EXCEPT ...", "input_image": ["/abs/path/problem.jpg"], "output_image": ["/abs/path/reasoning_01.jpg", "..."], "num_tokens": [144, 100, 81, ...] } ``` `num_tokens` is the per-image incremental token budget: the first image always uses the base budget `n_base`; each subsequent image uses a routed budget. There are two ways to supply it: - **Precomputed (default)** — if samples already carry `num_tokens`, pass `--use_json_num_tokens` (this is the default `TOKEN_ARGS` in `infer_deltav.sh`). - **TSIM Router (on the fly)** — otherwise set `TOKEN_ARGS="--use_tsim_router --visual_extractor_repo --visual_extractor_ckpt "`. The router (`deltav/tsim_tok/tsim_router.py`) measures temporal visual change with a frozen **DINOv2 ViT-B/14** and maps it to budgets via `tools/data_processing/tsim_intervals.json`. DINOv2 is fetched automatically by `torch.hub` on first use; to run offline, clone it and pass the local paths (see [INSTALL.md](INSTALL.md)). > **TSIM-Tok training and reconstruction evaluation are not included in this release.** The tokenizer *model* is retained as DeltaV's visual backbone, but its training/testing code will be added back later — see [TODO / Roadmap](#todo--roadmap). ## Documentation - [docs/eval_vlmevalkit.md](docs/eval_vlmevalkit.md): understanding benchmarks via VLMEvalKit. This guide is still being refined. The following guides will be published alongside the training / evaluation code (see [TODO / Roadmap](#todo--roadmap)): - [ ] `docs/data_and_token.md`: dataset format and the offline TSIM Router pipeline that turns image similarity into per-image token budgets. - [ ] `docs/eval_zebra_struct.md`: Zebra-CoT and StructCoT scoring. - [ ] `docs/advanced_zero3_gc.md`: ZeRO-3 and gradient checkpointing. - [ ] `docs/packing.md`: sequence packing, length computation, and packing training. ## Qualitative Examples

Qualitative comparison of multimodal reasoning

Qualitative comparison of multimodal reasoning. Full-image modeling (Base) exhibits inconsistent intermediate visual states, while DeltaV maintains consistent visual representations through visual state updates.

## Benchmark ### External Multimodal Reasoning and Understanding Evaluation

Model	#Param	VStar	EMMA	M3CoT	MathVista	VisuLogic	MMBench	MME‑P	MMVP
*General ULMMs*
Chameleon	7B	32.5	8.6	16.1	21.7	4.5	6.0	530	4.7
Anole	7B	34.0	6.6	15.8	22.5	3.7	6.2	508	6.7
Janus-pro	1B	43.5	18.9	45.9	37.6	25.0	60.2	1398	39.3
Janus-pro	7B	39.3	21.5	49.1	42.7	17.5	66.7	1509	34.7
OmniGen2	7B	41.4	14.7	50.3	60.2	0.1	76.1	1588	35.3
Bagel	7B	70.1	28.7	31.4	72.5	28.9	83.7	1665	69.3
EMU3.5	34B	-	-	-	28.3	11.4	13.7	791	16.7
*Understanding-centric MLLMs*
Qwen3-VL	2B	71.7	22.2	53.0	61.1	11.5	77.1	1482	45.0
Qwen3-VL	8B	83.7	30.6	61.2	77.6	22.5	85.2	1729	59.3
InternVL3.5	2B	68.1	12.7	51.3	60.8	26.0	78.2	1552	48.7
InternVL3.5	8B	69.1	16.6	59.9	74.1	29.7	82.7	1688	57.3
*Latent Interleaved Reasoning Models*
Monet	7B	79.1	22.1	44.2	62.5	10.6	75.3	1636	48.7
Mirage	8B	13.6	13.9	1.08	29.9	0.4	12.3	549	0.0
VPT-Det	2B	43.5	20.1	44.4	41.8	25.6	73.3	1516	34.0
*Explicit Interleaved Reasoning ULMMs*
Bagel-Zebra-CoT	7B	64.9	20.6	62.6	72.1	0	55.6	1647	22.0
ThinkMorph	7B	64.4	22.4	48.8	67.8	6.5	78.2	1478	8.6
DeltaV [Weight]	2B	75.9	28.6	54.5	69.3	23.5	82.3	1555	51.3

VStar, EMMA, M3CoT, MathVista, and VisuLogic are grouped as multimodal reasoning benchmarks, while MMBench, MME‑P, and MMVP are grouped as multimodal understanding benchmarks. ### In-domain Multimodal Reasoning Evaluation

Model	#Param	Zebra-CoT					StructCoT
Model	#Param	2D	3D	Science	Strategy	Overall	Strategy Planning	Spatial Planning	Logic	Math	Science	Visual Search	Jigsaw Restoration	Overall
*Understanding-centric MLLMs*
GPT-5.2	-	67.6	19.3	73.3	54.4	53.7	43.1	33.8	42.1	76.3	50.4	87.0	57.1	55.7
Gemini-3.1 Pro	-	68.7	19.0	83.3	60.4	57.9	71.6	28.2	50.2	78.3	55.0	79.4	65.3	61.1
Gemini 3.0 Flash	-	66.5	19.4	78.4	54.5	54.7	55.0	33.3	44.8	74.8	48.4	83.6	64.9	57.8
Qwen3-VL	2B	44.3	13.2	30.3	9.2	24.3	3.4	31.4	4.6	41.4	29.4	80.8	39.3	32.9
Qwen3-VL	8B	50.7	16.9	56.0	22.7	36.6	21.6	25.4	13.1	59.3	39.3	83.8	46.5	41.3
InternVL3.5	8B	29.7	11.4	48.9	19.8	27.5	6.9	36.3	17.5	36.1	32.0	75.8	41.0	35.1
Qwen2.5-VL	72B	43.2	17.3	50.1	25.8	34.1	14.8	34.4	31.4	48.0	36.5	84.9	47.0	42.4
*General ULMMs*
Chameleon	7B	13.3	3.0	5.2	9.9	7.9	5.6	12.5	4.1	9.1	13.1	23.5	14.4	11.8
Anole	7B	10.8	2.8	4.8	8.5	6.7	5.4	0.1	3.8	8.9	12.8	16.8	11.4	9.9
Janus-pro	7B	31.7	7.7	11.5	18.0	17.2	4.3	24.4	13.4	16.6	12.0	74.6	33.9	25.6
OmniGen2	7B	26.5	1.3	9.6	9.7	11.8	0.6	25.3	1.5	8.4	10.1	78.1	28.5	21.8
Bagel	7B	43.3	14.7	44.5	16.3	29.7	16.4	24.9	12.8	49.0	35.5	84.6	49.0	38.9
EMU3.5	34B	10.1	3.6	8.6	11.8	8.5	2.8	29.1	4.6	19.3	15.6	21.1	18.8	15.9
*Latent Interleaved Reasoning Models*
Monet	7B	37.5	12.0	15.1	23.0	21.9	2.3	19.9	21.9	33.8	25.8	59.6	33.8	28.1
Mirage	8B	2.2	2.5	10.7	12.4	7.0	0.9	14.3	12.4	35.8	22.5	11.0	30.4	18.2
VPT-Det	2B	32.3	3.5	6.5	18.7	15.3	7.5	26.5	8.8	14.5	15.1	73.1	35.9	25.9
*Explicit Interleaved Reasoning ULMMs*
Bagel-Zebra-CoT	7B	-	-	-	-	-	7.0	24.6	22.8	33.3	27.3	81.0	41.9	34.0
ThinkMorph	7B	43.0	11.6	31.4	22.9	27.2	21.4	19.5	26.4	43.4	26.0	84.1	49.9	38.7
DeltaV [Weight]	2B	78.9	20.0	41.1	38.3	44.6	16.4	53.0	66.0	30.1	45.6	84.3	62.6	51.1

The StructCoT test set excludes all samples originating from the Zebra-CoT dataset. ## Acknowledgements We would like to thank [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL) and [VFMTok](https://github.com/CVMI-Lab/VFMTok) for providing base models and code, as well as their contributions to this field. We also thank [Zebra-CoT](https://huggingface.co/datasets/multimodal-reasoning-lab/Zebra-CoT) for providing a valuable interleaved multimodal reasoning dataset. We also thank everyone who contributed to this open-source effort. ## Copyright Please do not hesitate to share your valuable feedback—it is a key motivation that drives us to continuously improve our framework. **Note:** Our model is intended for academic research and non-commercial use only. If you are interested in a faster (smaller) or stronger model, please contact us at [xbai@hust.edu.cn](mailto:xbai@hust.edu.cn) or [ylliu@hust.edu.cn](mailto:ylliu@hust.edu.cn).

如果您是本模型的贡献者，我们邀请您根据模型贡献文档，及时完善模型卡片内容。