---
title: cosmos-openvid-1m
canonical_url: "https://www.modelscope.cn/datasets/fal/cosmos-openvid-1m"
md_url: "https://www.modelscope.cn/datasets/fal/cosmos-openvid-1m.md"
repository: fal/cosmos-openvid-1m
last_updated: 2025-07-08
license: apache-2.0
storage_size: "2.1 TB"
downloads: 350
stars: 0
---

# cosmos-openvid-1m

> cosmos-openvid-1m - fal 在 ModelScope 开源的数据集。Cosmos-Tokenized OpenVid-1M

fal/cosmos-openvid-1m 是 ModelScope 魔搭社区上的数据集，存储大小 2.1 TB，采用 apache-2.0 许可。

- **Repository**: fal/cosmos-openvid-1m
- **License**: apache-2.0
- **Storage size**: 2.1 TB
- **Downloads**: 350
- **Stars**: 0
- **Last updated**: 2025-07-08

Source: https://www.modelscope.cn/datasets/fal/cosmos-openvid-1m

---

# Cosmos-Tokenized OpenVid-1M

[Cosmos-Tokenized](https://github.com/NVIDIA/Cosmos-Tokenizer) [OpenVid-1M](https://huggingface.co/datasets/nkp37/OpenVid-1M)

# How to use

Shards are stored in parquet format.
It has 4 columns: `serialized_latent`, `caption`, `fps`, `video`.

- `serialized_latent` is the latent vector of the video, serialized using `torch.save()`.
  Please use the following function to deserialize it:
  ```python
  def deserialize_tensor(
      serialized_tensor: bytes, device: Optional[str] = None
  ) -> torch.Tensor:
      return torch.load(
          io.BytesIO(serialized_tensor),
          weights_only=True,
          map_location=torch.device(device) if device else None,
      )
  ```
- `caption` is the caption of the video.
- `fps` is the fps of the video.
- `video` is the name of the video, you can find the original video at [OpenVid-1M](https://huggingface.co/datasets/nkp37/OpenVid-1M) dataset.

Example code to read the shards:

```python
import io
import json
from typing import Optional

import torch
import pandas as pd


def read_shards(type: str, split: str):
    # type: "discrete" or "continuous"
    index = json.load(open(f"{type}/{split}/index.json"))

    for shard in index["shards"]:
        shard_name = shard["raw_data"]["basename"]
        yield pd.read_parquet(f"{type}/{split}/{shard_name}")


def deserialize_tensor(
    serialized_tensor: bytes, device: Optional[str] = None
) -> torch.Tensor:
    return torch.load(
        io.BytesIO(serialized_tensor),
        weights_only=True,
        map_location=torch.device(device) if device else None,
    )


for shard in read_shards("discrete", "train"):
    for i, row in shard.iterrows():
        latent = deserialize_tensor(row["serialized_latent"])
        caption = row["caption"]
        fps = row["fps"]

        print(latent.shape)
        print(caption)
        print(fps)

```

To decode, you would need to install cosmos tokenizer.

```bash
git clone https://github.com/NVIDIA/Cosmos-Tokenizer.git
cd Cosmos-Tokenizer
apt-get install -y ffmpeg
pip install -e .
```

Download pretrained checkpoints.

```python
from huggingface_hub import login, snapshot_download

def download_pretrained_ckpts(local_dir: str, model_name: str):
    """Download pretrained checkpoints from huggingface."""

    login()
    os.makedirs(local_dir, exist_ok=True)
    snapshot_download(repo_id=f"nvidia/{model_name}", local_dir=local_dir)
```

Refer to the below code for getting the decoder.

```python
from cosmos_tokenizer.video_lib import CausalVideoTokenizer

def get_decoder(model_name: str  = "Cosmos-Tokenizer-DV4x8x8"):
    """Get the decoder for the given model name.
    model_name can be "Cosmos-Tokenizer-DV4x8x8", "Cosmos-Tokenizer-DV8x8x8", or "Cosmos-Tokenizer-DV8x16x16"."""

    local_dir = f"./pretrained_ckpts/{model_name}"
    if not os.path.exists(local_dir):
        download_pretrained_ckpts(local_dir, model_name)
    decoder = CausalVideoTokenizer(checkpoint_dec=f"{local_dir}/decoder.jit")
    return decoder
```

You need to unclamp the video to get it in range [0..255]. Decoded video is in range [-1,1].

```python
import torch
import numpy as np

_UINT8_MAX_F = float(torch.iinfo(torch.uint8).max)

def unclamp_video(input_tensor: torch.Tensor) -> torch.Tensor:
    """Unclamps tensor in [-1,1] to video(dtype=np.uint8) in range [0..255]."""
    tensor = (input_tensor.float() + 1.0) / 2.0
    tensor = tensor.clamp(0, 1).cpu().numpy()
    return (tensor * _UINT8_MAX_F + 0.5).astype(np.uint8)
```

Example code to decode and save the video with its caption.

```python
from torchvision.io import write_video

output_dir = "./output"
decoder = get_decoder()

for shard in read_shards("discrete", "train"):
    for i, row in shard.iterrows():
        latent = deserialize_tensor(row["serialized_latent"])
        caption = row["caption"]
        fps = row["fps"]

        # Squeeze/unsqueeze because the decoder expects a batch of videos.
        decoded_video = decoder.decode(latent.unsqueeze(0)).squeeze(0)

        # [C, T, H, W] -> [T, H, W, C]
        video = decoded_video.permute(1, 2, 3, 0)

        # Unclamp the video to get it in range [0..255].
        video = unclamp_video(video)

        # Write the video to disk.
        write_video(os.path.join(output_dir, f"{i:09d}.mp4"), video, fps=fps)

        # Write the caption to disk.
        with open(os.path.join(output_dir, f"{i:09d}.json"), "w") as f:
            json.dump({"caption": caption, "fps": fps}, f)
```