---
title: qth-embedding
canonical_url: "https://www.modelscope.cn/datasets/twinkle-kit/qth-embedding"
md_url: "https://www.modelscope.cn/datasets/twinkle-kit/qth-embedding.md"
repository: twinkle-kit/qth-embedding
chinese_name: "问题思维链相似度数据集"
last_updated: 2026-06-19
license: "Apache License 2.0"
downloads: 15
stars: 0
---

# qth-embedding

> qth-embedding - twinkle-kit 在 ModelScope 开源的数据集。QTH-Embedding Training Dataset

twinkle-kit/qth-embedding 是 ModelScope 魔搭社区上的数据集，采用 Apache License 2.0 许可。

- **Repository**: twinkle-kit/qth-embedding
- **License**: Apache License 2.0
- **Downloads**: 15
- **Stars**: 0
- **Last updated**: 2026-06-19

Source: https://www.modelscope.cn/datasets/twinkle-kit/qth-embedding

---

# QTH-Embedding Training Dataset

Pre-compressed embedding training dataset for reasoning-aware retrieval models. Each record contains structured summaries (anchor/positive/negative) produced by a condenser model, along with the original raw texts.

## Dataset Statistics

| Source | Rows | Description |
|--------|------|-------------|
| hard | 100,521 | Method-level hard negatives (abstract problem type + methodology) |
| think | 100,000 | Thinking traces (query + long CoT reasoning) |
| index | 100,000 | Reasoning solutions for RAG index construction |
| **Total** | **300,521** | |

## Schema

```json
{
  "anchor_text": "## Summary\nTopic: ...\nProblem: ...\nSkill: ...\nKnowledge: ...\n\n## More\n- ...",
  "positive_text": "## Summary\nTopic: ...\nProblem: ...\nSkill: ...\nKnowledge: ...\n\n## More\n- ...",
  "negative_texts": ["## Summary\n..."],
  "source": "hard | think | index",
  "query_raw": "original query text before compression",
  "cot_raw": "original CoT / positive passage before compression",
  "negs_raw": ["original negative passage(s)"]
}
```

| Field | Type | Description |
|-------|------|-------------|
| `anchor_text` | string | Compressed query — structured summary for embedding |
| `positive_text` | string | Compressed positive (CoT/passage) — structured summary |
| `negative_texts` | list[string] | Compressed hard negative(s); empty list if none |
| `source` | string | Data source category: `hard`, `think`, or `index` |
| `query_raw` | string | Original uncompressed query |
| `cot_raw` | string | Original uncompressed CoT / positive passage |
| `negs_raw` | list[string] | Original uncompressed negative passage(s) |

## Condenser

All texts are compressed using **Qwen3.5-4B-CM-v2** (`twinkle-kit/Qwen3.5-4B-CM-v2`), a fine-tuned condenser model based on Qwen3.5-4B. Inference is performed via vLLM (8×GPU, tensor parallel) with API fallback to `qwen3.7-max` for validation failures.

**Compression parameters:**
- Temperature: 0.2
- Top-p: 0.5
- Max output tokens: 8192

### Compression Strategy

The condenser applies different prompts depending on source type:

- **think / index**: Extract a retrieval-friendly structured summary preserving topic, problem, skill, and knowledge.
- **hard**: Extract the **abstract problem type** (ignoring specific numbers/values) and the **abstract methodology** (general technique and reasoning steps). This enables method-level clustering in embedding space.

### Output Format

All compressed texts follow the same skeleton:

```
## Summary
Topic: <subject — scope>
Problem: <what needs solving>
Skill: <method/technique required>
Knowledge: <theoretical concepts invoked>

## More
- <topic-key>: <expansion hint>
```

## Source Datasets

### Think (100,000 rows)

Sampled from long-form reasoning traces:

- Modotte/CodeX-2M-Thinking
- open-thoughts/OpenThoughts3-1.2M
- GAIR/LIMO-v2
- AI-ModelScope/Chinese-DeepSeek-R1-Distill-data-110k
- nohurry/Opus-4.6-Reasoning-3000x-filtered
- Roman1111111/claude-opus-4.6-10000x
- angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k

### Index (100,000 rows)

Sampled from diverse reasoning solutions:

- AI-ModelScope/OpenMathReasoning
- open-r1/OpenR1-Math-220k
- simplescaling/s1K-1.1
- nv-community/OpenCodeReasoning-2
- open-r1/codeforces-cots
- bespokelabs/Bespoke-Stratos-17k
- AI-ModelScope/dolphin-r1
- glaiveai/reasoning-v1-20m
- facebook/natural_reasoning
- HuggingFaceTB/cosmopedia
- AI-ModelScope/Omni-MATH

### Hard (100,521 rows)

Method-level hard negatives constructed from:

- AI-ModelScope/reasonir-data (hq subset) — provides query + positive + hard negative triplets
- xlangai/BRIGHT (documents corpus) — supplies the positive/negative passage texts

Each hard sample contains a query, a positive passage, and one hard negative passage. The three texts are independently compressed into abstract method descriptions, enabling the embedding model to learn fine-grained methodology discrimination.

## Intended Use

This dataset is designed for training embedding models with InfoNCE contrastive loss, where:
- `anchor_text` serves as the query embedding
- `positive_text` serves as the positive match
- `negative_texts` provide in-batch or explicit hard negatives

The `*_raw` fields are preserved for analysis, condenser retraining, and quality auditing.

## Production Pipeline

Built with `make_embedding_dataset.py` from the [Twinkle](https://github.com/tastelikefeet/twinkle) framework:

1. Load and cap source datasets (think ≤100k, index ≤100k, hard = all)
2. Compress via vLLM condenser (8 GPU, batch size 128)
3. Validate structure (`## Summary` + `## More` sections, 4-line template)
4. API fallback for validation failures
5. Incremental flush to JSONL with atomic progress tracking