---
title: Darwin-180B-NL2SQL
canonical_url: "https://www.modelscope.cn/models/FINAL-Bench/Darwin-180B-NL2SQL"
md_url: "https://www.modelscope.cn/models/FINAL-Bench/Darwin-180B-NL2SQL.md"
repository: FINAL-Bench/Darwin-180B-NL2SQL
last_updated: 2026-10-29
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
base_model:
  - FINAL-Bench/Darwin-180B-RSI
base_model_relation: finetune
library_name:
  - transformer
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 0
stars: 0
tags:
  - text-to-sql
  - nl2sql
  - bird
  - sqlite
  - darwin
---

# Darwin-180B-NL2SQL

> Darwin-180B-NL2SQL - FINAL-Bench 在 ModelScope 开源的模型。Darwin-180B-NL2SQL is VIDRAFT's text-to-SQL system built on Darwin-180B-RSI. It turns a natural-language question into a SQLite query, runs it against the database and revises the query from what the…

FINAL-Bench/Darwin-180B-NL2SQL 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 FINAL-Bench/Darwin-180B-RSI 构建。

- **Repository**: FINAL-Bench/Darwin-180B-NL2SQL
- **License**: apache-2.0
- **Tasks**: text-generation
- **Base model**: FINAL-Bench/Darwin-180B-RSI
- **Tags**: text-to-sql, nl2sql, bird, sqlite, darwin
- **Downloads**: 0
- **Stars**: 0
- **Last updated**: 2026-10-29

Source: https://www.modelscope.cn/models/FINAL-Bench/Darwin-180B-NL2SQL

---

# Darwin-180B-NL2SQL

<p align="center">
  <img src="https://img.shields.io/badge/BIRD_Mini--Dev_SQLite-83.80_EX-gold?style=for-the-badge" alt="BIRD Mini-Dev SQLite 83.80">
  <a href="https://huggingface.co/FINAL-Bench/Darwin-180B-RSI"><img src="https://img.shields.io/badge/Model-Darwin--180B--RSI-blue?style=for-the-badge" alt="Darwin-180B-RSI"></a>
  <a href="https://huggingface.co/FINAL-Bench"><img src="https://img.shields.io/badge/Org-FINAL--Bench-black?style=for-the-badge" alt="FINAL-Bench"></a>
</p>

Darwin-180B-NL2SQL is VIDRAFT's text-to-SQL system built on [Darwin-180B-RSI](https://huggingface.co/FINAL-Bench/Darwin-180B-RSI). It turns a natural-language question into a SQLite query, runs it against the database and revises the query from what the execution shows, then returns one final query per question.

This repository holds the system's predictions on BIRD Mini-Dev (SQLite) and the script used to score them.

> **Weights:** the model is [Darwin-180B-RSI](https://huggingface.co/FINAL-Bench/Darwin-180B-RSI), unchanged. Download the weights from that repository; no weights were changed or trained for this result.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "FINAL-Bench/Darwin-180B-RSI"  # weights for Darwin-180B-NL2SQL
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
```

## Result

**BIRD Mini-Dev, SQLite, 500 questions: 83.80% execution accuracy (419 / 500).**

| Difficulty | Correct | Accuracy |
|---|---|---|
| Challenging | 384 / 460 | 83.5% |
| Moderate | 29 / 32 | 90.6% |
| Simple | 6 / 8 | 75.0% |
| **All** | **419 / 500** | **83.80%** |

- Data: the current canonical release on Hugging Face, [`birdsql/bird_mini_dev`](https://huggingface.co/datasets/birdsql/bird_mini_dev), release "Mini-Dev 1024", `data/mini_dev_sqlite.jsonl` (sha256 `926af2a0d1b47e5d0d15a7aa8a34d4721007f4550a881555b6a89496b54a2606`), with the SQLite databases from `minidev.zip`.
- Metric: strict execution accuracy. A prediction is correct when the set of rows it returns equals the set of rows the reference SQL returns. Each predicted query may run for 60 seconds.
- Measured on 29 October 2026 (KST). The score was computed twice on an idle machine and was identical both times.

## Evaluation conditions

| Condition | Setting |
|---|---|
| Reference (gold) SQL used at any stage of inference | **No** |
| Oracle knowledge beyond the question and evidence given | No |
| Fine-tuning on BIRD | No |
| Final candidates per question | 1 |
| Database access during inference | Read-only execution of the system's own queries |

The question and evidence in each item are the only task inputs. The reference SQL was used only to score the final predictions.

## Files

| File | Contents |
|---|---|
| `predict_mini_dev_darwin-180b-nl2sql_SQLite.json` | Predictions in the BIRD Mini-Dev submission format (`SQL\t----- bird -----\tdb_id`, keyed by question ID) |
| `predictions.jsonl` | The same predictions, one JSON object per line |
| `score_ex.py` | Strict execution-accuracy scorer used for the numbers above |

To reproduce the score, place `mini_dev_sqlite.jsonl` next to `score_ex.py` and run:

```bash
python score_ex.py predictions.jsonl path/to/MINIDEV/dev_databases
```

## About the system

The system pairs Darwin-180B-RSI with VIDRAFT's NL2SQL verification engine, which checks each generated query against the live database before an answer is returned. The engine is VIDRAFT's own implementation and is not released.

A public demo of VIDRAFT's NL2SQL engine (Korean and English, read-only, synthetic data) is available at [final-bench-nl2sql-demo.hf.space](https://final-bench-nl2sql-demo.hf.space/).

## Contact

VIDRAFT · [vidraft.net](https://vidraft.net) · vidraft@mail.vidraft.net