---
title: humaneval-x
canonical_url: "https://www.modelscope.cn/datasets/ZhipuAI/humaneval-x"
md_url: "https://www.modelscope.cn/datasets/ZhipuAI/humaneval-x.md"
repository: ZhipuAI/humaneval-x
last_updated: 2024-08-19
license: "[apache-2.0]"
storage_size: "1.7 MB"
domain:
  - language
  - language_creators
  - pretty_name
  - task_ids
  - multilinguality
  - task_categories
  - size_categories
tasks:
  - code
  - expert-generated
  - HumanEval-X
  - language-modeling
  - multilingual
  - text-generation
  - unknown
downloads: 2957
stars: 1
---

# humaneval-x

> humaneval-x - ZhipuAI 在 ModelScope 开源的数据集。Dataset Description HumanEval-X is a benchmark for evaluating the multilingual ability of code generative models. It consists of 820 high-quality human-crafted data samples (each with test cases) in Python, C++,…

ZhipuAI/humaneval-x 是 ModelScope 魔搭社区上的code、expert-generated、HumanEval-X数据集，涉及 language、language_creators、pretty_name 领域，存储大小 1.7 MB，采用 [apache-2.0] 许可。

- **Repository**: ZhipuAI/humaneval-x
- **License**: [apache-2.0]
- **Tasks**: code, expert-generated, HumanEval-X, language-modeling, multilingual, text-generation, unknown
- **Domain**: language, language_creators, pretty_name, task_ids, multilinguality, task_categories, size_categories
- **Storage size**: 1.7 MB
- **Downloads**: 2957
- **Stars**: 1
- **Last updated**: 2024-08-19

Source: https://www.modelscope.cn/datasets/ZhipuAI/humaneval-x

---

# HumanEval-X

## Dataset Description
[HumanEval-X](https://github.com/THUDM/CodeGeeX) is a benchmark for evaluating the multilingual ability of code generative models. It consists of 820 high-quality human-crafted data samples (each with test cases) in Python, C++, Java, JavaScript, and Go, and can be used for various tasks, such as code generation and translation.

## Languages

The dataset contains coding problems in 5 programming languages: Python, C++, Java, JavaScript, and Go.

## Dataset Structure
To load the dataset you need to specify a subset among the 5 exiting languages  `[python, cpp, go, java, js]`. By default `python` is loaded. 

```python
from datasets import load_dataset
load_dataset("THUDM/humaneval-x", "js")

DatasetDict({
    test: Dataset({
        features: ['task_id', 'prompt', 'declaration', 'canonical_solution', 'test', 'example_test'],
        num_rows: 164
    })
})
```

```python
next(iter(data["test"]))
{'task_id': 'JavaScript/0',
 'prompt': '/* Check if in given list of numbers, are any two numbers closer to each other than\n  given threshold.\n  >>> hasCloseElements([1.0, 2.0, 3.0], 0.5)\n  false\n  >>> hasCloseElements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3)\n  true\n  */\nconst hasCloseElements = (numbers, threshold) => {\n',
 'declaration': '\nconst hasCloseElements = (numbers, threshold) => {\n',
 'canonical_solution': '  for (let i = 0; i < numbers.length; i++) {\n    for (let j = 0; j < numbers.length; j++) {\n      if (i != j) {\n        let distance = Math.abs(numbers[i] - numbers[j]);\n        if (distance < threshold) {\n          return true;\n        }\n      }\n    }\n  }\n  return false;\n}\n\n',
 'test': 'const testHasCloseElements = () => {\n  console.assert(hasCloseElements([1.0, 2.0, 3.9, 4.0, 5.0, 2.2], 0.3) === true)\n  console.assert(\n    hasCloseElements([1.0, 2.0, 3.9, 4.0, 5.0, 2.2], 0.05) === false\n  )\n  console.assert(hasCloseElements([1.0, 2.0, 5.9, 4.0, 5.0], 0.95) === true)\n  console.assert(hasCloseElements([1.0, 2.0, 5.9, 4.0, 5.0], 0.8) === false)\n  console.assert(hasCloseElements([1.0, 2.0, 3.0, 4.0, 5.0, 2.0], 0.1) === true)\n  console.assert(hasCloseElements([1.1, 2.2, 3.1, 4.1, 5.1], 1.0) === true)\n  console.assert(hasCloseElements([1.1, 2.2, 3.1, 4.1, 5.1], 0.5) === false)\n}\n\ntestHasCloseElements()\n',
 'example_test': 'const testHasCloseElements = () => {\n  console.assert(hasCloseElements([1.0, 2.0, 3.0], 0.5) === false)\n  console.assert(\n    hasCloseElements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3) === true\n  )\n}\ntestHasCloseElements()\n'}
 ```


## Data Fields

*   ``task_id``: indicates the target language and ID of the problem. Language is one of ["Python", "Java", "JavaScript", "CPP", "Go"].
*   ``prompt``: the function declaration and docstring, used for code generation.
*   ``declaration``: only the function declaration, used for code translation. 
*   ``canonical_solution``: human-crafted example solutions.
*   ``test``: hidden test samples, used for evaluation.
*   ``example_test``: public test samples (appeared in prompt), used for evaluation. 

## Data Splits

Each subset has one split: test.

## Citation Information

Refer to https://github.com/THUDM/CodeGeeX.