--- title: ArabCulture_verified canonical_url: "https://www.modelscope.cn/datasets/IFM/ArabCulture_verified" md_url: "https://www.modelscope.cn/datasets/IFM/ArabCulture_verified.md" repository: IFM/ArabCulture_verified last_updated: 2026-10-16 license: cc-by-nc-sa-4.0 storage_size: "642 KB" downloads: 76 stars: 0 --- # ArabCulture_verified > ArabCulture_verified - IFM 在 ModelScope 开源的数据集。ArabCulture (verified) IFM/ArabCulture_verified 是 ModelScope 魔搭社区上的数据集，存储大小 642 KB，采用 cc-by-nc-sa-4.0 许可。 - **Repository**: IFM/ArabCulture_verified - **License**: cc-by-nc-sa-4.0 - **Storage size**: 642 KB - **Downloads**: 76 - **Stars**: 0 - **Last updated**: 2026-10-16 Source: https://www.modelscope.cn/datasets/IFM/ArabCulture_verified --- # ArabCulture (verified) A filtered copy of **ArabCulture**, the Modern Standard Arabic benchmark of culturally grounded commonsense reasoning across 13 Arab countries, with samples flagged by a multi-judge audit removed. Each sample gives a short scenario and three candidate completions, exactly one of which is marked correct. Thirteen configs, one per country, each with a single `test` split: `Algeria`, `Egypt`, `Jordan`, `KSA`, `Lebanon`, `Libya`, `Morocco`, `Palestine`, `Sudan`, `Syria`, `Tunisia`, `UAE`, `Yemen`. > **What "verified" means.** Samples were removed when all three LLM judges flagged a significant defect. This is not a human review, and only samples that baseline models answered incorrectly were audited, so some defects may remain in the kept data. ## Source - Original dataset: [`MBZUAI/ArabCulture`](https://huggingface.co/datasets/MBZUAI/ArabCulture), revision `44927cec2abced759c70403d177d5c9c08410323` (files `/test-00000-of-00001.parquet`, one per config). The data files at this revision are identical to those at `08b9aac2cf93887e8a8040d42bdebb9bf52673b4`, the last upstream commit that changed them; later commits only touched the README and a figure. - No rows were added or edited. The only change is the removal of the excluded rows; every other row is kept as it appears in the source, including leading and trailing whitespace in the text fields. - The upstream annotation columns `should_discard` and `relevant_to_this_country` were **not** used for filtering and are kept as they are (see below). ## Columns Same columns, in the same order, as the source: - `worker_id`, `sample_id`: annotator and sample identifiers from the source (e.g. `Yemen-1`, `2 | Breakfast`). `sample_id` is not unique within most configs. - `sub_topic`: topic of the scenario, in Arabic. - `first_statement`: the scenario (question stem). - `options`: a struct of three parallel lists: `arabic_keys` (always `أ`, `ب`, `ج`), `english_keys` (always `A`, `B`, `C`) and `text` (the three completions, in that order). - `answer_key`: a struct with the gold letter in both scripts, `english_answer_key` (`A`/`B`/`C`) and `arabic_answer_key` (`أ`/`ب`/`ج`); the two always agree. It points to the entry of `options.text` at the same position. - `relevant_to_this_country`, `relevant_to_other_countries`, `should_discard`: annotator judgements from the source (`Yes`/`No` strings, with inconsistent leading spaces such as `" Yes"`, and some nulls). - `country`, `region`: the config's country and its region (`Gulf`, `Levant`, `North Africa`, `Nile Valley`). ### Upstream annotations not applied The source's own annotation columns were not used to filter this copy, and rows are kept regardless of their values. In the kept rows, 6 have `should_discard` = `" Yes"` (with a leading space) and 210 have `relevant_to_this_country` = `"No"` (18 more have it null). Values in these columns are not normalised (for example `should_discard` takes the values `"No"`, `" No"` and `" Yes"`). Users who want to apply these annotations should strip whitespace before comparing. ## Audit and filtering Samples were screened with the multi-judge audit described in [`IFM/discarded_samples`](https://huggingface.co/datasets/IFM/discarded_samples) (revision `3a3c1339`, config `ArabCulture`). Three language-model judges (GPT-5.6 Sol, Gemini 3.8 Flash, Claude Opus 5) assessed candidate samples that strong baselines answered incorrectly. A sample is excluded only when **all three** judges report a *significant* issue (a problem with the question, the options or the gold label that prevents reliable answering or scoring). Each discarded record carries a lowercase country name (`subset`) and a row index (`doc_id`); `doc_id` is the 0-based row index within that country's source file listed above (indices restart at 0 for each country). Every one of the 73 excluded rows was checked against its discarded record: the question text, all three options and the gold letter match after trimming whitespace (the discarded records store trimmed text), and no other row in the same country file matches. All other rows are kept unchanged and in their original order, with the original column names and values. Rows that every baseline answered correctly were not candidates for the audit, so they were not reviewed and may still contain issues. | Config | Original rows | Excluded | Kept | |---|---:|---:|---:| | `Algeria` | 271 | 4 | 267 | | `Egypt` | 265 | 3 | 262 | | `Jordan` | 290 | 5 | 285 | | `KSA` | 261 | 6 | 255 | | `Lebanon` | 255 | 11 | 244 | | `Libya` | 239 | 9 | 230 | | `Morocco` | 276 | 11 | 265 | | `Palestine` | 273 | 2 | 271 | | `Sudan` | 256 | 2 | 254 | | `Syria` | 279 | 2 | 277 | | `Tunisia` | 261 | 14 | 247 | | `UAE` | 283 | 2 | 281 | | `Yemen` | 273 | 2 | 271 | | **Total** | **3482** | **73** | **3409** | ## License The source dataset is released under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NC-SA 4.0), and this filtered copy is distributed under the same license. See the [original dataset card](https://huggingface.co/datasets/MBZUAI/ArabCulture) for details. ## Citation Please cite the original authors: ```bibtex @inproceedings{sadallah-etal-2025-commonsense, title = "Commonsense Reasoning in {A}rab Culture", author = "Sadallah, Abdelrahman and Tonga, Junior Cedric and Almubarak, Khalid and Almheiri, Saeed and Atif, Farah and Qwaider, Chatrine and Kadaoui, Karima and Shatnawi, Sara and Alesh, Yaser and Koto, Fajri", booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)", month = jul, year = "2025", address = "Vienna, Austria", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.acl-long.380/", doi = "10.18653/v1/2025.acl-long.380", pages = "7695--7710" } @misc{jais3_2026, title = {Jais 3: Introducing Arabic-centric Agentic AI}, author = {{IFM Team, Inception42 Team, and Cerebras Team}}, year = {2026}, url = {https://ifm.ai/blog/jais3/}, } ```