---
title: PSVU-10K
canonical_url: "https://www.modelscope.cn/datasets/YuanhongZheng/PSVU-10K"
md_url: "https://www.modelscope.cn/datasets/YuanhongZheng/PSVU-10K.md"
repository: YuanhongZheng/PSVU-10K
last_updated: 2026-11-17
license: cc-by-nc-4.0
storage_size: "15 GB"
downloads: 0
stars: 0
---

# PSVU-10K

> PSVU-10K - YuanhongZheng 在 ModelScope 开源的数据集。PSVU-10K is the training set of PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts. It contains 10k verified four-option questions about user-defined concepts in causal video stream…

YuanhongZheng/PSVU-10K 是 ModelScope 魔搭社区上的数据集，存储大小 15 GB，采用 cc-by-nc-4.0 许可。

- **Repository**: YuanhongZheng/PSVU-10K
- **License**: cc-by-nc-4.0
- **Storage size**: 15 GB
- **Downloads**: 0
- **Stars**: 0
- **Last updated**: 2026-11-17

Source: https://www.modelscope.cn/datasets/YuanhongZheng/PSVU-10K

---

# PSVU-10K

PSVU-10K is the training set of [PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts](https://arxiv.org/abs/2603.20422). It contains 10k verified four-option questions about user-defined concepts in causal video stream prefixes. PEARL uses it to train the query side of its clip retriever and its answering VLM with LoRA.

- Code: <https://github.com/Yuanhong-Zheng/PEARL>
- Benchmark: [PSVU-Bench](https://huggingface.co/datasets/zyh200727/PSVU-Bench)
- Mirror: [ModelScope](https://www.modelscope.cn/datasets/YuanhongZheng/PSVU-10K)

## Statistics

| Split | Real-Time | Past-Time | Total | Source videos |
| --- | ---: | ---: | ---: | ---: |
| `train` | 3,616 | 5,384 | 9,000 | 142 |
| `validation` | 359 | 641 | 1,000 | 13 |
| **Total** | **3,975** | **6,025** | **10,000** | **155** |

- Training and validation questions come from disjoint source videos. The 155 source videos are 155 different narrative works, mostly movies.
- Each sample is a causal prefix with a median length of 7.5 minutes (1 to 15 minutes), split into 8-second clips.
- Each sample registers two to four concepts (median four).
- Past-Time evidence ends a median of 32 seconds and up to 6.5 minutes before the question.
- 6,021 of the 6,025 Past-Time questions have one or two verified negative clips. Real-Time questions have none.
- PSVU-10K contains entity concepts only (no Action concepts) and shares no work with PSVU-Bench.

## Files

```text
PSVU-10K/
  README.md
  LICENSE
  train.jsonl   # 9,000 records
  val.jsonl     # 1,000 records
  PSVU-10K-train/                 # ready-to-train snapshot (15.9 GB)
    samples.jsonl                 # the 10,000 samples in the format read by the training code
    document_embeddings.pt        # precomputed Qwen3-VL-Embedding-2B vectors of all 63,339 clips
    release_text_queries.json     # cached rewritten retrieval queries
    manifest.json, dataset_manifest.json, release_provenance.json
    frames-000.tar ... frames-002.tar   # 549,376 frames at 1 FPS, 13.2 GB
    SHA256SUMS
```

`source_video` is an anonymous identifier that groups records from the same video. The full source videos are not distributed. Instead, `PSVU-10K-train/` contains the 1 FPS frames of every clip that the questions use (stream prefixes, concept reference frames, evidence and negative clips), so the released training code runs on it directly. In `samples.jsonl`, `source_id` equals `source_video`, `sample_id` equals `id`, and frame paths are relative to the snapshot root.

### Training with the snapshot

```bash
huggingface-cli download zyh200727/PSVU-10K --repo-type dataset --include "PSVU-10K-train/*" --local-dir PSVU-10K
cd PSVU-10K/PSVU-10K-train
sha256sum -c SHA256SUMS
for f in frames-*.tar; do tar -xf "$f"; done   # creates frames/; the tar files can then be deleted

cd /path/to/PEARL   # https://github.com/Yuanhong-Zheng/PEARL
pip install -r requirements-train.txt && pip install -e training --no-deps
bash scripts/train_retriever.sh /path/to/PSVU-10K/PSVU-10K-train runs/psvu10k   # writes runs/psvu10k/retrieval
bash scripts/train_reader.sh    /path/to/PSVU-10K/PSVU-10K-train runs/psvu10k   # writes runs/psvu10k/reader
```

The base models default to `models/Qwen3-VL-Embedding-2B` and `models/Qwen3-VL-8B-Instruct` in the repository; set `EMBEDDING_MODEL` or `READER_MODEL` to use other paths. The cached queries and embeddings are tied to the released code (commit `9c3c763` or later with unchanged training files).

## Record format

Each line is one JSON record. All times are in seconds on the timeline of the source video, and every clip is a `[start, end]` pair.

```json
{
  "id": "psvu10k-e72152a142acab04",
  "split": "val",
  "type": "Past-Time",
  "question": "Along the reed-lined path, who lifts a bicycle to his shoulder first?",
  "options": ["{Miro}", "{Lumi}", "{Zavi}", "An unregistered person"],
  "answer": "C",
  "answer_index": 2,
  "concepts": [
    {"name": "Miro", "description": "the man on the right facing the camera, wearing a light grey patterned jacket ...", "reference_time": 6559.019333},
    {"name": "Lumi", "description": "the bald older man with a grey mustache ...", "reference_time": 6557.019333},
    {"name": "Zavi", "description": "the man on the left seen from behind, wearing a blue jacket and glasses", "reference_time": 6559.019333}
  ],
  "evidence_clips": [[6557, 6565], [6565, 6573]],
  "context_clips": [[6557, 6565], [6565, 6573]],
  "negative_clips": [[6589, 6597], [6605, 6613]],
  "current_clip": [6613, 6621],
  "source_video": "psvu10k-video-097",
  "prefix_start": 6301,
  "question_time": 6621
}
```

| Field | Description |
| --- | --- |
| `id` | Unique record identifier. |
| `split` | `train` or `val`. |
| `type` | `Real-Time` (asks about the current clip) or `Past-Time` (asks about earlier evidence). |
| `question` | Question text. Registered concepts appear in braces, e.g. `{Miro}`. |
| `options` | The four answer options. They include registered names and distractors such as "No one" or "An unregistered person". |
| `answer` | Letter of the correct option (`A`-`D`). |
| `answer_index` | 0-based index of the correct option. |
| `concepts` | Registered concepts, each with its `name`, a short `description`, and the `reference_time` of the reference frame in which the concept is clearly visible. Every reference time lies inside the prefix and before the question. |
| `evidence_clips` | Clips that contain the evidence for the answer. These are the positives for retriever training. For Real-Time questions this is the current clip. |
| `context_clips` | Clips shown to the answering model as supporting history during training. Usually equal to `evidence_clips`. |
| `negative_clips` | Clips that the verifier confirmed do not support the answer. These are the negatives for retriever training. |
| `current_clip` | The last clip of the prefix, which ends at `question_time`. |
| `source_video` | Identifier of the source video. Records with the same identifier come from the same video. |
| `prefix_start` | Start of the visible stream prefix. |
| `question_time` | Time at which the question is asked, i.e. the end of the prefix. |

The prefix from `prefix_start` to `question_time` is a contiguous sequence of 8-second clips. Its last clip is `current_clip`, and all clips listed in a record lie on this grid.

## Construction

Qwen3.8-Flash-Next annotates the characters and events in the last 64 seconds of each prefix and writes one four-option question at its end. A question is accepted only if it passes all of the following checks, each run with shuffled options:

1. After reviewing the whole prefix in chunks, without any hint of the evidence location, the verifier selects exactly the annotated answer.
2. It answers correctly from the evidence clips and the concept references alone.
3. It answers correctly from the training context, namely the evidence clips plus at most two clips that it has confirmed not to support the answer.
4. It cannot answer once the visual evidence is withheld, or once the concept definitions are removed.

Concept names are randomly sampled. Every work that appears in PSVU-Bench was excluded using the work name in each video's metadata, so training never sees the characters on which PEARL is evaluated. See the paper for details.

## Sources

PSVU-10K is built from YouTube videos of 155 narrative works, mostly movies. Candidate works were ranked by their metadata with GPT-6 Luna and kept when the same characters recur, which Qwen3.8-Flash-Next confirmed by inspecting frames.

## Intended use

PSVU-10K is intended for non-commercial research on personalized and streaming video understanding, in particular for training and validating concept-aware clip retrievers and answering models. The validation split is for model selection, not for reporting benchmark results. Use PSVU-Bench for evaluation. The dataset is not intended for identifying real people or for any surveillance application.

## License

The annotations and metadata are released under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) (see `LICENSE`). The frames in `PSVU-10K-train/` come from third-party videos, which remain the property of their respective owners and are subject to their original terms. They are provided for non-commercial research only.

## Privacy and removal requests

The questions and concept descriptions describe on-screen characters in publicly available narrative works. Characters are referred to only by randomly sampled names. The records contain no personal information about the people shown.

If you own content used in PSVU-10K, appear in it, or have any other concern, email **zyh2022200727@gmail.com** with the record `id` or `source_video` and the reason for the request. We will review the request and remove the affected records from future versions of the dataset.

## Citation

```bibtex
@article{zheng2026pearl,
  title={PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts},
  author={Zheng, Yuanhong and An, Ruichuan and Lin, Xiaopeng and Liu, Yuxing and Yang, Sihan and Zhang, Huanyu and Li, Haodong and Zhang, Qintong and Zhang, Renrui and Li, Guopeng and Zhang, Yifan and Li, Yuheng and Zhang, Wentao},
  journal={arXiv preprint arXiv:2603.20422},
  year={2026},
  archivePrefix={arXiv},
  eprint={2603.20422},
  primaryClass={cs.CV}
}
```
