For more, download the dataset to explore the full benchmark locally.
## Download EPIC-Bench contains ~35k small files across three task categories. To work around per-file rate limit and to keep download speed reasonable, the annotations are distributed as **three task-level tarballs** instead of raw folders. Download the tarballs from [HuggingFace](https://huggingface.co/datasets/rxc205/EPIC-Bench) (or [ModelScope](https://www.modelscope.cn/datasets/macarich/EPIC-Bench)): | Archive | Size | # Files | Contents | |---------|------|---------|----------| | `Manipulation.tar.gz` | 1.70 GB | 7,061 | AffordanceRegion / ContactRelationship / PlacementRegion | | `Navigation.tar.gz` | 2.71 GB | 10,868 | FeasiblePath / GroundDetection / VisualMatching | | `TargetLocalization.tar.gz` | 3.34 GB | 17,665 | BasicAttributes / EmbodiedCompositionalAttributes / SpatialRelatedAttributes | Place them under `dataset/annotation/` and extract: ```bash mkdir -p dataset/annotation/EPIC_Bench cd dataset/annotation/EPIC_Bench # Option 1: huggingface-cli hf download rxc205/EPIC-Bench \ --repo-type dataset --local-dir . # Option 2: modelscope modelscope download \ --dataset macarich/EPIC-Bench \ --local_dir . # Extract all three (preserves the original folder layout) for f in Manipulation.tar.gz Navigation.tar.gz TargetLocalization.tar.gz; do tar -xzf "$f" && rm "$f" done ``` ## 🧰 EPIC-Bench Evaluation Toolkit We provide [evaluation code](https://github.com/rxc205/EPIC-Bench-Eval#-epic-bench-evaluation-toolkit) for both open-source VLMs via **ms-swift** and API-based VLMs via **lmms-eval**. For details, please refer to our GitHub repository. ## 📰 News - [2026.5.28] 🎉 Over 1,000 downloads across [HuggingFace](https://huggingface.co/datasets/rxc205/EPIC-Bench) and [ModelScope](https://www.modelscope.cn/datasets/macarich/EPIC-Bench). Thanks for the community’s support! - [2026.5.19] 🚀 Our [arXiv paper](https://arxiv.org/abs/2605.17070) is now available! - [2026.5.15] 🚀 [HuggingFace](https://huggingface.co/datasets/rxc205/EPIC-Bench) and [ModelScope](https://www.modelscope.cn/datasets/macarich/EPIC-Bench) Dataset are available! - [2026.5.15] 🚀 [Project Page](https://epic-bench.github.io/EPIC-Bench/) and [Evaluation Code](https://github.com/rxc205/EPIC-Bench-Eval) are released, the arXiv paper will come soon. ## 📋 Todo - [x] Evaluation code for EPIC-Bench - [x] The EPIC-Bench datasets - [ ] Make the evaluation pipeline compatible with mask outputs ## 🏆 Leaderboard and Benchmark Please refer to the [EPIC-Bench Homepage](https://epic-bench.github.io/EPIC-Bench/) for the full leaderboard, dataset downloads, and data examples. ## 📬 Contact With Us - Email: xiancong.ren@x-humanoid.com ## 📚 Citation ```BibTeX @misc{shan2026epicbenchperceptioncentricbenchmarkfinegrained, title={EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models}, author={Haozhe Shan and Xiancong Ren and Han Dong and Haoyuan Shi and Yingji Zhang and Jiayu Hu and Yi Zhang and Yong Dai and Bin Shen and Lizhen Qu and Zenglin Xu and Xiaozhu Ju}, year={2026}, eprint={2605.17070}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2605.17070}, } ``` ## 📜 License This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/rxc205/EPIC-Bench-Eval/blob/main/LICENSE) file for details. ## 🙏 Acknowledgements - **ms-swift** for open-source VLM inference: [ms-swift](https://swift.readthedocs.io/zh-cn/latest/) - **lmms-eval** for API/closed-source evaluation: [lmms-eval](https://github.com/EvolvingLMMs-Lab/lmms-eval)