---
title: vibe-design-arena
canonical_url: "https://www.modelscope.cn/datasets/datapointai/vibe-design-arena"
md_url: "https://www.modelscope.cn/datasets/datapointai/vibe-design-arena.md"
repository: datapointai/vibe-design-arena
last_updated: 2026-08-24
license: cc-by-4.0
storage_size: "1.2 GB"
downloads: 95
stars: 0
---

# vibe-design-arena

> vibe-design-arena - datapointai 在 ModelScope 开源的数据集。The first public pairwise human preference dataset for real-world vibe-coded web applications.

datapointai/vibe-design-arena 是 ModelScope 魔搭社区上的数据集，存储大小 1.2 GB，采用 cc-by-4.0 许可。

- **Repository**: datapointai/vibe-design-arena
- **License**: cc-by-4.0
- **Storage size**: 1.2 GB
- **Downloads**: 95
- **Stars**: 0
- **Last updated**: 2026-08-24

Source: https://www.modelscope.cn/datasets/datapointai/vibe-design-arena

---

# Vibe Design Arena

The first public pairwise human preference dataset for real-world vibe-coded web applications.

60 apps from the [Vibe Coding Showcase](https://vibecodingshowcase.com/) were compared pairwise by human annotators who judged which app has better visual design based on screenshots. Every possible pair was evaluated (C(60,2) = 1,770 comparisons), with 30 human votes per pair.

## Dataset Summary

| Stat | Value |
|------|-------|
| Apps | 60 |
| Pairwise comparisons | 1,770 |
| Human votes per comparison | 30 |
| Total human judgments | ~53,000 |
| Screenshot resolution | 1440×900 |

## Subsets

### `apps` (60 rows)

One row per app with metadata and aggregate rankings.

| Field | Type | Description |
|-------|------|-------------|
| `app_id` | string | Unique identifier (derived from screenshot filename) |
| `app_name` | string | Project name |
| `website` | string | Live URL |
| `description` | string | Project description |
| `tools` | list[string] | Normalized vibe coding tools (e.g. `["Bolt", "Cursor"]`) |
| `frameworks` | list[string] | Frameworks and libraries used |
| `build_time_raw` | string | Original build time text (e.g. "2 weeks") |
| `build_time_days` | float | Normalized build time in days |
| `revenue` | string | Revenue information (self-reported) |
| `creator` | string | Creator name |
| `location` | string | Creator location |
| `screenshot` | image | 1440×900 screenshot of the app |
| `wins` | int | Number of pairwise matchups won (out of 59) |
| `losses` | int | Number of pairwise matchups lost |
| `win_rate` | float | wins / (wins + losses) |
| `rank` | int | 1–60, sorted by win rate |

### `comparisons` (1,770 rows)

One row per pairwise comparison with vote counts and both screenshots.

| Field | Type | Description |
|-------|------|-------------|
| `app_a` | string | App ID of first app |
| `app_b` | string | App ID of second app |
| `image_a` | image | Screenshot of app A |
| `image_b` | image | Screenshot of app B |
| `votes_a` | int | Number of votes for app A (out of 30) |
| `votes_b` | int | Number of votes for app B (out of 30) |
| `winner` | string | `"app_a"` or `"app_b"` |

## Usage

```python
from datasets import load_dataset

# Load app metadata and rankings
apps = load_dataset("datapointai/vibe-design-arena", "apps")

# Load pairwise comparisons
comparisons = load_dataset("datapointai/vibe-design-arena", "comparisons")
```

## Key Findings

### Tool Rankings (by aggregate win rate)
1. **Bolt** — 58.6% win rate (21 apps)
2. **Cursor** — 57.3% win rate (27 apps)
3. **Claude** — 51.3% win rate (16 apps)
4. **Lovable** — 45.1% win rate (13 apps)
5. **ChatGPT** — 42.2% win rate (13 apps)
6. **Replit** — 45.3% win rate (8 apps)

### Build Time vs Design Quality
No meaningful correlation (Pearson r = 0.23). The #2 ranked app (ActivDNA, 95% win rate) was built in 3 days.

### Revenue
Only 3 of 60 apps report actual revenue. No correlation between design quality and revenue.

## What Makes This Dataset Unique

- **Real shipped products** — not AI-generated from synthetic prompts
- **Complete pairwise matrix** — every app compared against every other app
- **Rich metadata** — tools, frameworks, build time, revenue, creator info
- **Large-scale annotation** — 30 votes per pair, ~53K total human judgments
- **Vibe coding focused** — all apps built with AI-assisted development tools

## Comparison to Related Work

| Dataset | Type | Apps | Pairs | Votes | Real Products? |
|---------|------|:----:|:-----:|:-----:|:--------------:|
| **Vibe Design Arena** | Pairwise preference | 60 | 1,770 | ~53K | Yes |
| UI-Bench | Pairwise preference | 300 | — | 4K+ | No (AI-generated) |
| DesignPref | Pairwise preference | — | 12K | 12K | No (generated) |
| UICrit | Critique/rating | 983 | — | 3K | No (mobile UIs) |

## Data Collection

Apps were sourced from the [Vibe Coding Showcase](https://vibecodingshowcase.com/) hackathon. Screenshots were captured programmatically at 1440×900 resolution using Playwright. Pairwise comparisons were conducted on [Datapoint AI](https://datapoint.ai/), where human annotators viewed two screenshots side-by-side and selected the one with better visual design. Each comparison received 30 independent votes.

## License

CC-BY-4.0

## Citation

```bibtex
@dataset{vibe_design_arena_2025,
  title={Vibe Design Arena: A Pairwise Human Preference Dataset for Vibe-Coded Web App Design},
  author={Datapoint AI},
  year={2025},
  url={https://huggingface.co/datasets/datapointai/vibe-design-arena}
}
```