---
title: SynthCypher
canonical_url: "https://www.modelscope.cn/datasets/ServiceNow-AI/SynthCypher"
md_url: "https://www.modelscope.cn/datasets/ServiceNow-AI/SynthCypher.md"
repository: ServiceNow-AI/SynthCypher
last_updated: 2025-03-20
license: "Apache License 2.0"
storage_size: "17 MB"
downloads: 1287
stars: 1
---

# SynthCypher

> SynthCypher - ServiceNow-AI 在 ModelScope 开源的数据集。SynthCypher Dataset Repository

ServiceNow-AI/SynthCypher 是 ModelScope 魔搭社区上的数据集，存储大小 17 MB，采用 Apache License 2.0 许可。

- **Repository**: ServiceNow-AI/SynthCypher
- **License**: Apache License 2.0
- **Storage size**: 17 MB
- **Downloads**: 1287
- **Stars**: 1
- **Last updated**: 2025-03-20

Source: https://www.modelscope.cn/datasets/ServiceNow-AI/SynthCypher

---

# SynthCypher Dataset Repository

## Overview

This repository hosts **SynthCypher**, a novel synthetic dataset designed to bridge the gap in **Text-to-Cypher (Text2Cypher)** tasks. SynthCypher leverages state-of-the-art **large language models (LLMs)** to automatically generate and validate high-quality data for training and evaluating models that convert natural language questions into Cypher queries for graph databases like Neo4j. 

Our dataset and pipeline contribute significantly to advancing Text2Cypher research by offering a large, diverse, and rigorously validated dataset across a wide range of query types and domains.

---

## Highlights of SynthCypher

- **Comprehensive Coverage**: 
  - 25.8k training samples and 4k test samples.
  - Spanning **109 query types** (e.g., Simple Retrieval, Aggregation, Sub-Graph Queries).
  - Derived from **528 training schemas** and **165 testing schemas**.

- **Synthetic Data Generation Pipeline**: 
  - **Schema Generation**: 700 diverse domains expanded using **Mixtral**.
  - **Natural Language Question Creation**: 109 query types with corresponding dummy ground truths.
  - **Neo4j Database Population**: Populated with synthetic data to validate schema and Cypher queries.
  - **Cypher Query Generation**: Iterative chain-of-thought reasoning by LLMs for high-quality query generation.
  - **Validation**: Rigorous validation of Cypher execution and correctness using LLMs and Neo4j.

- **Performance Gains**:
  - LLMs fine-tuned on SynthCypher achieve **40% improvement** over baseline datasets and outperform off-the-shelf models.

---

## Dataset Details

The dataset consists of:
- **Schemas**: Representing real-world domains (e.g., e-commerce, inventory).
- **Natural Language Questions**: Diverse queries crafted for each schema.
- **Cypher Queries**: High-quality queries aligned with natural language questions.

---

## Experimental Results

Key observations from our experiments:
1. **Performance Gap**: Existing models trained on generic instruction datasets show low accuracy on Text2Cypher tasks.
2. **SynthCypher Effectiveness**: Fine-tuning with SynthCypher improves model performance by up to 40% absolute over baseline datasets.
3. **Controlled Data Generation**: Our pipeline demonstrates superior quality and coverage compared to naive GPT-based approaches.

---

## Limitations

- **Synthetic Data Bias**: Synthetic strategies may not fully reflect real-world distributions and could reinforce biases.
- **Real-World Applicability**: Performance on real-world scenarios may vary.

---

## Citation

If you use SynthCypher in your work, please cite:

```
@misc{tiwari2024synthcypherfullysyntheticdata,
      title={Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework}, 
      author={Aman Tiwari and Shiva Krishna Reddy Malay and Vikas Yadav and Masoud Hashemi and Sathwik Tejaswi Madhusudhan},
      year={2024},
      eprint={2412.12612},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.12612}, 
}
```

---

## License

This dataset is licensed under the *Creative Commons Attribution Non Commercial Share Alike 4.0*. Please review the terms before use.

---