---
title: Cultural-Evaluation-Kalahi
canonical_url: "https://www.modelscope.cn/datasets/aisingapore/Cultural-Evaluation-Kalahi"
md_url: "https://www.modelscope.cn/datasets/aisingapore/Cultural-Evaluation-Kalahi.md"
repository: aisingapore/Cultural-Evaluation-Kalahi
last_updated: 2026-01-07
license: "[cc-by-4.0]"
storage_size: "43 KB"
downloads: 75
stars: 0
---

# Cultural-Evaluation-Kalahi

> Cultural-Evaluation-Kalahi - aisingapore 在 ModelScope 开源的数据集。Kalahi evaluates the ability of LLMs to generate responses relevant to Filipino culture in terms of shared knowledge and ethics. This dataset contains a MCQ-compatible version of the Kalahi dataset…

aisingapore/Cultural-Evaluation-Kalahi 是 ModelScope 魔搭社区上的数据集，存储大小 43 KB，采用 [cc-by-4.0] 许可。

- **Repository**: aisingapore/Cultural-Evaluation-Kalahi
- **License**: [cc-by-4.0]
- **Storage size**: 43 KB
- **Downloads**: 75
- **Stars**: 0
- **Last updated**: 2026-01-07

Source: https://www.modelscope.cn/datasets/aisingapore/Cultural-Evaluation-Kalahi

---

# Kalahi

Kalahi evaluates the ability of LLMs to generate responses relevant to Filipino culture in terms of shared knowledge and ethics. This dataset contains a MCQ-compatible version of the [Kalahi](https://arxiv.org/abs/2409.15380) dataset that is used in [SEA-HELM](https://leaderboard.sea-lion.ai/).

### Supported Tasks and Leaderboards

Kalahi is designed for evaluating Filipino cultural representations in instruction-tuned large language models (LLMs). It is part of the [SEA-HELM](https://leaderboard.sea-lion.ai/) leaderboard from [AI Singapore](https://aisingapore.org/).

### Languages
- Tagalog (tl)

### Dataset Details
Kalahi only has a Tagalog (tl) split. Below are the statistics for this dataset. The number of tokens only refer to the strings of text found within the `prompts` column.

| Split | # of examples | # of GPT-4o tokens | # of Gemma 2 tokens | # of Llama 3 tokens |
|-|:-|:-|:-|:-|
| tl | 150 | 23710 | 26534 | 29766 |

### Data Sources

| Data Source | License | Language/s | Split/s
|-|:-|:-| :-|
| [Kalahi](https://huggingface.co/datasets/aisingapore/kalahi) | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Tagalog | tl

### License

For the license/s of the dataset/s, please refer to the data sources table above.

We endeavor to ensure data used is permissible and have chosen datasets from creators who have processes to exclude copyrighted or disputed data. 


## Acknowledgement

This project is supported by the National Research Foundation Singapore and Infocomm Media Development Authority (IMDA), 
Singapore under its National Large Language Model Funding Initiative.

### References

```bibtex
@misc{montalan2024kalahihandcraftedgrassrootscultural,
      title={Kalahi: A handcrafted, grassroots cultural LLM evaluation suite for Filipino}, 
      author={Jann Railey Montalan and Jian Gang Ngui and Wei Qi Leong and Yosephine Susanto and Hamsawardhini Rengarajan and William Chandra Tjhi and Alham Fikri Aji},
      year={2024},
      eprint={2409.15380},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2409.15380}, 
}
```