---
title: BlenderWorld
canonical_url: "https://www.modelscope.cn/datasets/Qwen/BlenderWorld"
md_url: "https://www.modelscope.cn/datasets/Qwen/BlenderWorld.md"
repository: Qwen/BlenderWorld
last_updated: 2026-10-15
license: mit
storage_size: "589 MB"
downloads: 108281
stars: 9
---

# BlenderWorld

> BlenderWorld - Qwen 在 ModelScope 开源的数据集。BlenderWorld BlenderWorld is the first comprehensive benchmark dedicated to evaluating frontier models on realistic 3D creation tasks. BlenderWorld comprises 223 tasks spanning architecture, products, natural…

Qwen/BlenderWorld 是 ModelScope 魔搭社区上的数据集，存储大小 589 MB，采用 mit 许可。

- **Repository**: Qwen/BlenderWorld
- **License**: mit
- **Storage size**: 589 MB
- **Downloads**: 108281
- **Stars**: 9
- **Last updated**: 2026-10-15

Source: https://www.modelscope.cn/datasets/Qwen/BlenderWorld

---

# BlenderWorld
BlenderWorld is the first comprehensive benchmark dedicated to evaluating frontier models on realistic 3D creation tasks. BlenderWorld comprises 223 tasks spanning architecture, products, natural environments, and scientific visualization, driven by text, image, video, and structured specification inputs. To rigorously assess these complex deliverables, a unified evaluation protocol is established: The submitted Blender projects are rendered from multiple viewpoints and evaluated against 2,651 task-specific binary rubrics alongside holistic visual quality scores. For more details, please refer to our Project: [BlenderWorld](https://github.com/QwenLM/BlenderWorld).

# BlenderWorld-Harbor

The benchmark comprises 223 tasks and 2,651 rubric criteria. Each instance includes a Base64-encoded image preview and corresponding JSON data.

The package contains 78 text, 130 image-and-text, and 15 video-and-text tasks.
Use **Harbor 0.23.0**, Python 3.12+ and a GPU-capable Linux sandbox.
The commands below use the Modal backend. [中文使用说明](使用说明.md)

## Task categories

The 223 tasks span **9 categories and 22 subcategories**. Use [taxonomy.csv](taxonomy.csv)
for the per-task mapping, [taxonomy.json](taxonomy.json) for the hierarchy, counts and
figure labels, or [classification_summary.md](classification_summary.md) for a readable table.
Each task also includes `category`, `subcategory`, `category_short`, `subcategory_short`
and `taxonomy_version` in `task.toml` metadata and its evaluator dataset record.

## View tasks and embedded images

Extract the ZIP and open `index.html` in a browser. Search by task ID or query,
or filter by modality. Each task's `environment/inputs/instruction.html` displays
its query, Base64-embedded reference images and declared attachments.
Videos play from the bundled files using relative paths.

All 130 input images are available in `environment/inputs/reference_images.base64.json`.
Each image record includes `path`, `media_type`, `bytes`, `sha256`, `width`, `height`
and a complete `data_uri`. Tasks without input images have an empty `images` array.
The one verifier-only image is in `bw_0182/tests/rubric_reference_images.base64.json`.

```python
import base64, hashlib, json

with open("bw_0006/environment/inputs/reference_images.base64.json") as f:
    image = json.load(f)["images"][0]
raw = base64.b64decode(image["data_uri"].split(",", 1)[1], validate=True)
assert hashlib.sha256(raw).hexdigest() == image["sha256"]
```

Base64 decoding returns the full image file bytes. `base64_manifest.json` lists all
image locations and fingerprints. Standard image files are also available for
Harbor agents and the evaluator. The agent's `instruction.md` contains a short text
prompt and file paths; encoded image payloads are in the HTML/JSON files.

## Run with Harbor

```bash
python -m pip install "harbor[modal]==0.23.0"
cd BlenderWorld-harbor-base64
sha256sum -c SHA256SUMS
```

Configure Modal credentials with `modal token new`, or use your existing Modal
credentials. You can select another GPU-capable Harbor backend with `--env`.
Harbor 0.23.0's built-in Docker backend does not allocate GPUs from `task.toml`;
use a backend that supports the declared GPU requirement.

Configure the generation agent's provider credentials. For the verifier, set the
environment variables documented in `verifier.env.example`:

| Variable | Purpose |
| --- | --- |
| `BW_AGENT_JUDGE_ENDPOINT` | Anthropic-compatible endpoint for rendering/scene-inspection agents |
| `BW_AGENT_JUDGE_API_KEY` | Key for that endpoint |
| `BW_VISION_JUDGE_ENDPOINT` | Full Chat Completions endpoint for the vision judge |
| `BW_VISION_JUDGE_API_KEY` | Key for the vision endpoint |
| `BW_RENDER_MODEL` | Optional rendering-agent model alias |
| `BW_AGENT_JUDGE_MODEL` | Optional scene-inspection model alias |
| `BW_VISION_JUDGE_MODEL` | Optional vision-judge model alias |

Use model aliases supported by your endpoints. Then run a task or the full dataset:

```bash
harbor run -p ./bw_0006 --env modal -a codex --ak version=0.147.0 -m 'provider/model'
harbor run -p . --env modal -a codex --ak version=0.147.0 -m 'provider/model' --n-concurrent 1
```

For Claude Code, use `-a claude-code --ak version=2.1.220`. Set `-p` to a task
directory or the package directory. `dataset.toml` is the publication manifest.

## Inputs, outputs and scoring

The agent reads `/workspace/inputs/` and writes its editable project to
`/workspace/exports/final.blend`. Keep requested images, videos, model exports
and texture dependencies inside `/workspace/exports/` with relative paths.

Harbor transfers the exports to a separate verifier container. The verifier
packages a standard `bw_XXXX.zip` submission and uses the BlenderWorld evaluator.
`/logs/verifier/reward.json` contains `reward`, `rubric_score` and `dimension_score`
on a 0–1 scale. Multiply mean `reward` by 100 for the overall score. Detailed
scores and evidence are in `metrics.json` and `run/`.

To inspect a collected submission using installed BlenderWorld tools:

```bash
bw evaluate --dataset ./bw_0006/tests/dataset \
  --submission /path/to/bw_0006.zip --output /path/to/new-evaluation --dry-run
```

Configure the runtime and judge endpoints, then omit `--dry-run` to render and score.

## Layout and runtime

```text
BlenderWorld-harbor-base64/
  index.html
  README.md
  使用说明.md
  dataset.toml
  base64_manifest.json
  export_manifest.json
  validation.json
  verifier.env.example
  SHA256SUMS
  bw_0001/
    instruction.md
    task.toml
    environment/
      Dockerfile
      inputs/
        query.txt
        input.json
        instruction.html
        reference_images.base64.json
        references/                # when present
        attachments/               # when present
    tests/
      Dockerfile
      test.sh
      blenderworld/
      dataset/
  ... bw_0223/
```

Runtime: Blender 4.2.10, Node.js 22.17.0, Codex CLI 0.147.0, Claude Code 2.1.220,
MCP for Blender 2.1.3. The generation environment and the separate verifier
environment each request 8 CPUs, 16 GiB RAM, 100 GiB storage and **1 GPU**.
The package does not restrict the GPU model. Use hardware compatible with Blender
Cycles; configure CUDA or OptiX when rendering on NVIDIA GPUs.
The generation timeout is **8 hours (28,800 seconds) per task**. The verifier has
its own **8-hour** timeout. These are separate phase budgets; environment build
timeout is 1 hour. The GPU requirement is specified in both `[environment]` and
`[verifier.environment]` in every `task.toml`.
Use a generation agent; no oracle solution is bundled. The MIT `LICENSE` covers
the adapter/evaluator code. Use `SHA256SUMS` to check package integrity and
`validation.json` to inspect the available validation results.