---
title: Common-O
canonical_url: "https://www.modelscope.cn/datasets/facebook/Common-O"
md_url: "https://www.modelscope.cn/datasets/facebook/Common-O.md"
repository: facebook/Common-O
last_updated: 2026-03-12
license: mit
storage_size: "365 MB"
downloads: 233
stars: 0
---

# Common-O

> Common-O - facebook 在 ModelScope 开源的数据集。measuring multimodal reasoning across scenes

facebook/Common-O 是 ModelScope 魔搭社区上的数据集，存储大小 365 MB，采用 mit 许可。

- **Repository**: facebook/Common-O
- **License**: mit
- **Storage size**: 365 MB
- **Downloads**: 233
- **Stars**: 0
- **Last updated**: 2026-03-12

Source: https://www.modelscope.cn/datasets/facebook/Common-O

---

# Common-O

> measuring multimodal reasoning across scenes


Common-O, inspired by cognitive tests for humans, probes multimodal LLMs' ability to reason across scenes by asking "what’s in common?"

![fair conference content copy.001](https://cdn-uploads.huggingface.co/production/uploads/64c17345e82e55936cf971bc/5av7avUrsBjFuMrWuOiCW.jpeg)

Common-O is comprised of household objects:


![fair conference content copy.003](https://cdn-uploads.huggingface.co/production/uploads/64c17345e82e55936cf971bc/hEvVz2uFR6z-jv1em25eY.jpeg)


We have two subsets: Common-O (3 - 8 objects) and Common-O Complex (8 - 16 objects).

## Multimodal LLMs excel at single image perception, but struggle with multi-scene reasoning


![single_vs_multi_image(1)](https://cdn-uploads.huggingface.co/production/uploads/64c17345e82e55936cf971bc/1cB9iXHrSgyvfXgK6gmGu.png)


## Evaluating a Multimodal LLM on Common-O

```python
import datasets

# get a sample
common_o = datasets.load_dataset("facebook/Common-O")["main"]
# common_o_complex = datasets.load_dataset("facebook/Common-O")["complex"]
x = common_o[3]

output: str = model(x["image_1"], x["image_2"], x["question"])

check_answer(output, x["answer"])
```

To check the answer, we use an exact match criteria:

```python
import re

def check_answer(generation: str, ground_truth: str) -> bool:
    """
    Args:
        generation: model response, expected to contain "Answer: ..."
        ground_truth: comma-separated string of correct answers

    Returns: bool, whether the prediction matches the ground truth
    """
    preds = generation.split("\n")[-1]
    preds = re.sub("Answer:", "", preds)
    preds = preds.split(",")
    preds = [p.strip() for p in preds]
    preds = sorted(preds, key=lambda x: x[0])

    # split into a list
    ground_truth_list = [a.strip() for a in ground_truth.split(",")]
    ground_truth_list = sorted(ground_truth_list)
    return preds == ground_truth_list
```

Some models have specific formatting outputs for their answers, e.g. \boxed{A} or Answer: A. We recommend checking a few responses as you may notice slight variations based on this.
This public set also has slight variations with the set used in the original paper, so while the measured capabilities are identical do not expect an exact replication of accuracy figures.


If you'd like to use a single image model, here's a handy function to turn `image_1` and `image_2` into a single split image:

```python
from PIL import Image

def concat_images_horizontal(
        image1: Image.Image, image2: Image.Image, include_space: bool=True, space_width: int=20, fill_color: tuple=(0, 0, 0)
        )  -> Image.Image:
    # from https://note.nkmk.me/en/python-pillow-concat-images/
    if not include_space:
        dst = Image.new("RGB", (image1.width + image2.width, image1.height))
        dst.paste(image1, (0, 0))
        dst.paste(image2, (image1.width, 0))
    else:
        total_width = image1.width + space_width + image2.width
        max_height = max(image1.height, image2.height)

        dst = Image.new("RGB", (total_width, max_height), color=fill_color)
        dst.paste(image1, (0, (max_height - image1.height) // 2))
        dst.paste(image2, (image1.width + space_width, (max_height - image2.height) // 2))
    return dst
```

For more details about Common-O see the 
- [dataset card](https://huggingface.co/datasets/facebook/Common-O/blob/main/COMMON_O_DATASET_CARD.md)
- [ArXiv Paper](https://arxiv.org/abs/2511.03768)

Please note we fixed an upload issue as of March 3, 2026, so please consider using the latest version if you cached an earlier version. Thank you to Milad Afshari for catching this!

Cite:
```
@inproceedings{Ross2025what0s,
  title  = {What’s in Common? Multimodal Models Hallucinate When Reasoning Across Scenes},
  author = {Candace Ross and Florian Bordes and Adina Williams and Polina Kirichenko and Mark Ibrahim},
  year   = {2025},
  url    = {https://openreview.net/attachment?id=d0F0N0cu4n&name=supplementary_material}
}
```