---
title: Rexverse-2M-Camera
canonical_url: "https://www.modelscope.cn/datasets/AI-ModelScope/Rexverse-2M-Camera"
md_url: "https://www.modelscope.cn/datasets/AI-ModelScope/Rexverse-2M-Camera.md"
repository: AI-ModelScope/Rexverse-2M-Camera
last_updated: 2026-10-21
license: other
storage_size: "826 MB"
downloads: 151
stars: 0
---

# Rexverse-2M-Camera

> Rexverse-2M-Camera - AI-ModelScope 在 ModelScope 开源的数据集。Per-image camera parameter annotations for the Rexverse-2M dataset (the released image subset of IDEA-Research/Rexverse-2M; 415,857 images across 84 shards), captioned by the Puffin-World model. More…

AI-ModelScope/Rexverse-2M-Camera 是 ModelScope 魔搭社区上的数据集，存储大小 826 MB，采用 other 许可。

- **Repository**: AI-ModelScope/Rexverse-2M-Camera
- **License**: other
- **Storage size**: 826 MB
- **Downloads**: 151
- **Stars**: 0
- **Last updated**: 2026-10-21

Source: https://www.modelscope.cn/datasets/AI-ModelScope/Rexverse-2M-Camera

---

# Rexverse-2M-Camera

![camera map collage](analysis/Rexverse-2M-Camera.png)

Per-image camera parameter annotations for the **Rexverse-2M** dataset
(the released image subset of [IDEA-Research/Rexverse-2M](https://huggingface.co/datasets/IDEA-Research/Rexverse-2M); 415,857 images across 84 shards), captioned by the [**Puffin-World**](https://huggingface.co/ACERobotics/Puffin-World) model. More captioned datasets are provided in our [**Puffin-16M**](https://kangliao929.github.io/projects/puffin-16m/) website.

The collage above visualizes the camera maps on sample images — each
pair shows the **up field** (green arrows: the projected gravity-up direction)
and the **latitude field** (colored contours: angle above/below the horizon).

## Format

One `.tar` per shard (`rexverse_0000.tar` … ), each containing one `.json` per
image whose name matches the source image stem.

Each JSON holds the predicted monocular camera parameters:

| Field | Meaning | Unit |
|-------|---------|------|
| `roll`  | camera roll        | radians |
| `pitch` | camera pitch       | radians |
| `vfov`  | vertical field-of-view | radians |
| `k1`    | radial distortion coefficient | – |
| `parse_ok` | whether the model output parsed within valid ranges | bool |

Example:

```json
{"roll": 0.0123, "pitch": -0.0871, "vfov": 1.0123, "k1": 0.0000, "parse_ok": true}
```

## Camera Parameter Distributions

Histograms of the predicted roll / pitch / vertical-FoV over the whole dataset
(proportion of valid samples per 10° bin; `parse_ok=False` excluded).

![rexverse camera stats](analysis/rexverse_camera_stats.png)

| split | roll μ / med / σ | pitch μ / med / σ | FoV μ / med / σ |
|-------|------------------|-------------------|-----------------|
| all (414,634) | 0.2° / 0.0° / 6.8° | −4.0° / −0.9° / 12.9° | 32.2° / 28.9° / 11.1° |

- **Roll** is sharply peaked at 0° (web images shot upright/level).
- **Pitch** is centered slightly negative with a broad spread (σ ≈ 12.9°).
- **FoV** spans a wide range (σ ≈ 11.1°, median ≈ 29°) — consistent with the
  diverse focal lengths of large-scale web imagery.

If you'd like a dataset with a more diverse and uniform distribution of camera parameters, please refer to our [Puffin-4M](https://huggingface.co/datasets/KangLiao/Puffin-4M) and [Puffin-16M](https://huggingface.co/datasets/KangLiao/Puffin-16M) datasets.

### Dataset Download
You can download the entire dataset using the following command:
```bash
hf download KangLiao/Rexverse-2M-Camera --repo-type dataset
```

### From Camera Parameters to Up and Latitude Fields

The released `(roll, pitch, vfov, k1)` annotations can be converted into the dense perspective-field representation used by Puffin-World. The conversion computes focal length from `vfov`, constructs a radial camera, and maps roll and pitch to the gravity direction. [`get_perspective_field`](https://github.com/KangLiao929/Puffin/blob/main/Puffin-World/scripts/camera/geometry/perspective_fields.py) then returns a normalized 2-channel **up field** and a 1-channel **latitude field**. The up field encodes the projected world-up direction at every pixel, while the latitude field measures each viewing ray's angular elevation relative to the horizon.

```python
import json
import torch
from scripts.camera.geometry.camera import SimpleRadial
from scripts.camera.geometry.gravity import Gravity
from scripts.camera.geometry.perspective_fields import get_perspective_field
from scripts.camera.utils.conversions import fov2focal

with open("camera.json") as f:
    annotation = json.load(f)
roll, pitch, vfov, k1 = (
    annotation[key] for key in ("roll", "pitch", "vfov", "k1")
)

H, W = 512, 512
f = float(fov2focal(torch.tensor(vfov), H))
camera = SimpleRadial(torch.tensor(
    [W, H, f, f, W / 2, H / 2, k1, 0.0]
).float()).scale(torch.tensor([1.0, 1.0]))
gravity = Gravity.from_rp(torch.tensor(roll), torch.tensor(pitch))
up_field, latitude_field = get_perspective_field(camera, gravity)
# Shapes: [1, 2, H, W] and [1, 1, H, W]
```

Run the example from the `Puffin-World` directory. Use [`plot_vector_fields` and `plot_latitudes`](https://github.com/KangLiao929/Puffin/blob/main/Puffin-World/scripts/camera/visualization/viz2d.py) to render up-field arrows and latitude heatmaps or contours. See [`save_pf_visualization`](https://github.com/KangLiao929/Puffin/blob/main/Puffin-World/scripts/demo/physics_perception.py) for an end-to-end visualization example.

### Caption Pipeline
Beyond this captioned dataset, we also release **a complete captioning pipeline** for annotating camera parameters for arbitrary datasets, analyzing camera parameter distributions, and visualizing the corresponding camera maps. The pipeline is available in our [GitHub repository](https://github.com/KangLiao929/Puffin/blob/main/Puffin-World/documents/ANNOTATION_CAMERA.md).


### Citation
If you find the captioned dataset useful for your research or applications, please cite the following papers using these BibTeX entries:

```bibtex
  @article{liao2025puffin,
    title={Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation},
    author={Liao, Kang and Wu, Size and Wu, Zhonghua and Jin, Linyi and Wang, Chao and Wang, Yikai and Wang, Fei and Li, Wei and Loy, Chen Change},
    journal={arXiv preprint arXiv:2510.08673},
    year={2025}
  }

  @article{liao2026puffinworld,
    title={Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States},
    author={Liao, Kang and Luo, Yihang and Wu, Xiao-Ming and Jin, Linyi and Wu, Size and Lin, Chunyu and Zhao, Yao and Wang, Fei and Li, Wei and Loy, Chen Change},
    journal={arXiv preprint arXiv:2609.04196},
    year={2026}
  }
```