--- title: RynnVLA-Latent-4B canonical_url: "https://www.modelscope.cn/models/DAMO_Academy/RynnVLA-Latent-4B" md_url: "https://www.modelscope.cn/models/DAMO_Academy/RynnVLA-Latent-4B.md" repository: DAMO_Academy/RynnVLA-Latent-4B last_updated: 2026-10-08 license: apache-2.0 pipeline_tag: robotics tasks: - robotics model_type: - rynn_brain_vla architectures: - Qwen3VLForConditionalGeneration parameters: 5.0B tensor_type: - BF16 library_name: - pytorch - safetensors frameworks: - pytorch language: - en downloads: 10 stars: 0 tags: - vision-language-action - vla - latent-action - latent-action-model - robot-manipulation - flow-matching - mixture-of-transformers - embodied-ai - pytorch --- # RynnVLA-Latent-4B > RynnVLA-Latent-4B - DAMO_Academy 在 ModelScope 开源的模型。Stage-1 latent-action pretraining checkpoint of RynnVLA-Latent, a Mixture-of-Transformers vision-language-action model. Given a language instruction and up to six camera views, it generates a chunk of… DAMO_Academy/RynnVLA-Latent-4B 是 ModelScope 魔搭社区上的 5.0B 参数robotics模型，采用 apache-2.0 许可。 - **Repository**: DAMO_Academy/RynnVLA-Latent-4B - **License**: apache-2.0 - **Tasks**: robotics - **Parameters**: 5.0B - **Tags**: vision-language-action, vla, latent-action, latent-action-model, robot-manipulation, flow-matching, mixture-of-transformers, embodied-ai, pytorch - **Downloads**: 10 - **Stars**: 0 - **Last updated**: 2026-10-08 Source: https://www.modelscope.cn/models/DAMO_Academy/RynnVLA-Latent-4B --- # RynnVLA-Latent-4B Stage-1 latent-action pretraining checkpoint of **RynnVLA-Latent**, a Mixture-of-Transformers vision-language-action model. Given a language instruction and up to six camera views, it generates a chunk of **latent actions** by flow matching. - Parameters: **5,042,270,129** (5.042 B), 1,426 tensors, all `bfloat16` - Weights: **EMA** snapshot at the final training step **236,442** - Frozen during Stage 1: the vision tower and the language model (parameter regexes `visual\.`, `language_model\.`) - Text backbone: hidden 2560, 36 layers, 32 attention heads / 8 KV heads, intermediate 9,728 - Exported: 2026-10-03 Code, data format and training scripts: [github.com/alibaba-damo-academy/RynnVLA-Latent](https://github.com/alibaba-damo-academy/RynnVLA-Latent) --- ## Read this first: what this checkpoint is *not* This is a **Stage-1** checkpoint. Its flow-matching target is the latent action, not a robot action, so Stage 1 consumes no action labels at all. As shipped, `processor_config.json` carries an empty action/state schema (`"schema": {"action": {}, "state": {}}`, `"use_state": false`), and the eval adapters in the repository explicitly refuse a Stage-1 checkpoint for closed-loop control — `validate_libero_schema` raises *"Use a Stage2 action checkpoint, not a Stage1 latent-only checkpoint"* whenever `use_latent_actions` is true. So: **do not point LIBERO / RoboTwin / VLABench evaluation at this repo.** Use it as the initialization for Stage 2, which keeps the same flow-matching objective and retargets it from latent actions to a robot-specific action space. `action_dim = 81` and `action_chunk_size = 30` are declared in the config and carried in the architecture, but Stage-1 training does not use them. ## Latent-action protocol (608-dim `ktoken_zcam`) Stage 1 trains against latent actions produced by [RynnLAM](https://modelscope.cn/models/DAMO_Academy/RynnLAM), which labels raw RGB frame pairs and needs no action space — that is what lets real-robot trajectories, simulation rollouts and human egocentric video share one target representation. | Field | Value | | --- | --- | | `latent_action_dim` | 608 = 8×64 k-tokens (512) + 64 `z` + 32 camera-pose | | `latent_action_chunk_size` | 6 | | `latent_action_stride` | 5 | | Labeling cadence | `gap = 4`, `pair_stride = 4` — one latent action per 4 RGB frames | ## Architecture | Config | Value | | --- | --- | | `model_type` | `rynn_brain_vla` | | `architectures` | `Qwen3VLForConditionalGeneration` | | `action_head_type` | `expert` | | `expert_backbone` | `qwen3_vl` | | `expert_hidden_size` | 768 | | `time_conditioning` | `adaln` | | `num_view_slots` | 6 (`head`, `left_wrist`, `right_wrist`, `front_third`, `side_left`, `side_right`) | | `use_view_cond_slots` | true | | Vision encoder | `qwen3_vl`, depth 24, hidden 1024, patch 16, spatial merge 2, deepstack layers [5, 11, 17] | | Tokenizer | Qwen3 BPE, 151,936 embedding rows; vocabulary identical to the public Qwen3-VL tokenizer (151,643 tokens / 151,387 merges) plus one extra special token `<|state_pad|>` | | `transformers_version` | 5.2.0 | Up to 6 camera roles are carried per sample with a `slot_mask` marking the valid ones, so a source with one camera and a source with four train in the same batch. Images are normalized with `image_mean = image_std = [0.5, 0.5, 0.5]`, i.e. rescaled to `[-1, 1]` (`processor_class = RynnBrainVLAProcessor`). The separate `action_norm_type` field is `mean_std`; it governs action/state normalization and is inert at Stage 1, where `schema.action` and `schema.state` are both empty. ## Loading The custom `model_type` is resolved by the `rynnvla` package, so import it first: ```python import torch from rynnvla.models.rynn_brain_vla import RynnBrainVLAModel, RynnBrainVLAProcessor path = "./RynnVLA-Latent-4B" # a local directory, e.g. from snapshot_download below model = RynnBrainVLAModel.from_pretrained( path, dtype=torch.bfloat16, attn_implementation="flash_attention_2", ) processor = RynnBrainVLAProcessor.from_pretrained(path) ``` Verified on `transformers==5.2.0` / `torch==2.7.1` with both `attn_implementation="eager"` and `"flash_attention_2"`: the checkpoint materializes with **zero** missing and zero unexpected keys, the parameter count equals the `safetensors` header count exactly (5,042,270,129), and no tensor contains a non-finite value. The only exactly-zero tensors are the three action-head biases (`state_proj.bias`, `action_in_proj.bias`, `action_out_proj.bias`), which Stage 1 does not train; their weights are non-zero. Download from ModelScope: ```python from modelscope import snapshot_download model_dir = snapshot_download("DAMO_Academy/RynnVLA-Latent-4B") ``` ## Files | File | Bytes | SHA-256 | | --- | --- | --- | | `LICENSE` | 11,358 | `cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30` | | `chat_template.jinja` | 5,292 | `3636d0f0bd6bef02654cdffdc447b79cb2cef8ab02cc75267345946291a489e4` | | `config.json` | 3,152 | `5a41a1e4d7d77eff8e420c9155331cd1aaa5fff2b12331073f94718ded3edebe` | | `export_provenance.json` | 1,275 | `8422f71f4ad19b1ccf8bb8525ed47fc7f7cea05b9c85de9cd3b6a4f0587695ab` | | `model.safetensors` | 10,084,715,450 | `7fd7c0c5a7687d72f4baed0379c13183269c40d49ba93d021a4245d6f5f71ac7` | | `processor_config.json` | 1,470 | `50074d83ffd4c389324e6e07bd9f3b889fadbbd9376941d303768d1990ea4ccc` | | `tokenizer.json` | 11,422,840 | `f9c0d33d111cb70baf6140ce788268a980ad8b6f70981c35b76193bfa5fb45aa` | | `tokenizer_config.json` | 495 | `fab563ef8444a432383b2e5ea31076699cbc0dddd3029b3e5884603f36a67f94` | `export_provenance.json` records the source step, the EMA selection, the frozen-parameter regexes, the source weight digest and the tensor count, plus an `export_file_sha256` manifest over the six sibling files. Internal absolute paths were removed from it before release, which changes only that file's own digest; the sibling manifest deliberately does not cover it, so all six recorded digests still match. ## Training Stage-1 latent-action pretraining on the full latent-action corpus, global batch size 512, run to completion at step 236,442. The released weights are the EMA snapshot of that final step, and the frozen trunk digest is recorded as `e577269c42fd307c…`. ## License Apache License 2.0. ## Links - Code & training scripts: [github.com/alibaba-damo-academy/RynnVLA-Latent](https://github.com/alibaba-damo-academy/RynnVLA-Latent) - RynnLAM (produces the 608-dim latent actions): [modelscope.cn/models/DAMO_Academy/RynnLAM](https://modelscope.cn/models/DAMO_Academy/RynnLAM) - Sibling checkpoints: [RynnVLA-Latent-2B](https://modelscope.cn/models/DAMO_Academy/RynnVLA-Latent-2B) · [RynnVLA-Latent-4B](https://modelscope.cn/models/DAMO_Academy/RynnVLA-Latent-4B)