--- title: Qwen3.5-4B-Condenser canonical_url: "https://www.modelscope.cn/models/twinkle-kit/Qwen3.5-4B-Condenser" md_url: "https://www.modelscope.cn/models/twinkle-kit/Qwen3.5-4B-Condenser.md" repository: twinkle-kit/Qwen3.5-4B-Condenser chinese_name: "Qwen3.5-4B上下文压缩模型" last_updated: 2026-05-14 license: apache-2.0 pipeline_tag: text-generation tasks: - text-generation base_model: - Qwen/Qwen3.5-4B base_model_relation: adapter library_name: - lora - safetensors - pytorch frameworks: - Pytorch language: - en downloads: 15843 stars: 0 tags: - condenser - compression - lora - sft - hotpotqa - long-context --- # Qwen3.5-4B-Condenser > Qwen3.5-4B-Condenser - twinkle-kit 在 ModelScope 开源的模型。HotpotQA Condenser — Qwen3.5-4B LoRA twinkle-kit/Qwen3.5-4B-Condenser 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 Qwen/Qwen3.5-4B 构建。 - **Repository**: twinkle-kit/Qwen3.5-4B-Condenser - **License**: apache-2.0 - **Tasks**: text-generation - **Base model**: Qwen/Qwen3.5-4B - **Tags**: condenser, compression, lora, sft, hotpotqa, long-context - **Downloads**: 15843 - **Stars**: 0 - **Last updated**: 2026-05-14 Source: https://www.modelscope.cn/models/twinkle-kit/Qwen3.5-4B-Condenser --- # HotpotQA Condenser — Qwen3.5-4B LoRA A passage-level **Condenser** LoRA, SFT-tuned from Qwen3.5-4B on the [HotpotQA Condensed 9k](https://www.modelscope.cn/datasets/twinkle-kit/hotpotqa-condensed-9k) dataset. The model rewrites a single passage into a two-section telegraphic markdown (`## Summary` + `## More`) so that a downstream multi-hop QA agent can "read the digest first, then decide whether to expand the original". Trained with [`cookbook/rl/train_condenser_ddp.py`](https://github.com/modelscope/twinkle) on top of the [Twinkle](https://github.com/modelscope/twinkle) framework. --- ## 1. I/O Contract Single-turn chat. The model is invoked with a system prompt + a user prompt; the assistant response is the compression. ### System prompt (verbatim, fixed) The same `CONDENSER_SYSTEM` used to produce the training data — verbatim: ````text You are a text compression assistant. A downstream model will read your compressed output to decide whether the detail it needs is inside this block; if yes, it will fetch and read the original passage. Downstream model workflow: Read your compressed output -> Decide whether needed info is in this block -> If yes -> Fetch original. Therefore your compression MUST NOT lose major information from the source. Output format: ```text ## Summary Overview plus facts STRONGLY RELATED to the Query, stated explicitly. ## More A collapsed index; expansion required to see specific information. ``` Rules: 1. Telegraphic style — drop function words ("the", "a", "is", "are", "of", ...); colons and commas mean "is" / "has". 2. Summary MUST contain the passage's primary topic + 2–4 concrete core facts drawn from the source (entities, numbers, dates, relations). If a Query is given, order Query-relevant facts first, but STILL include other core facts within the budget. A Query is an ORDERING HINT, NOT a filter. 3. Summary MUST NOT be meta-commentary about the Query. Forbidden patterns: "no X mention", "Query info: absent", "passage covers Y only", "does not contain ...", "no relevant info", or summaries that are only abstract category words like "structure/order/usage" with no facts. If the passage is unrelated to the Query, you still summarize the passage normally. 4. More is an INDEX of category keywords, NOT inline data. Enumerate what CAN be recovered from the source (e.g. "birthplace, death place, age"); do NOT paste dates/numbers/names inline. Make sure all category of useful facts are introduced here. 5. Output language MUST match the source language. 6. Do NOT fabricate. Do NOT omit major information. Any fact not in the source MUST NOT appear in your output. Example: Source: ```text Marie Curie (7 Nov 1867 – 4 Jul 1934), born Maria Sklodowska in Warsaw (then Russian Poland); parents were teachers. Barred from Polish universities, she and her sister agreed to take turns funding each other's overseas study. In 1891 Marie reached Paris and enrolled at the Sorbonne, earning a physics degree (1893) and a mathematics degree (1894), becoming the school's first female physics lecturer. In 1895 she married French physicist Pierre Curie; they spent the rest of their lives on radioactivity research. In July 1898 she discovered polonium, named after her homeland Poland; in December she and Pierre announced the discovery of radium. She coined "radioactivity" and showed it is an atomic property, not a chemical reaction. In 1903 she shared the Nobel Prize in Physics with Pierre and Henri Becquerel. In 1911 she alone won the Nobel Prize in Chemistry for polonium and radium. She is the first woman to win a Nobel, and the only person to win Nobels in two different sciences. After Pierre died in a carriage accident in 1906, Marie took his chair and became the first female professor at the Sorbonne. During World War I she developed mobile X-ray units, called "Petites Curies" in French; about 20 were deployed to the front, examining over 1,000,000 wounded soldiers. She died of aplastic anaemia from radiation exposure on 4 July 1934 in Passy, Haute-Savoie, France, aged 66. Her notebooks remain highly radioactive, kept in lead boxes; researchers must wear protective gear to consult them. ``` Compressed: ```text ## Summary Marie Curie: French-Polish physicist/chemist, founder of radioactivity research, first female Sorbonne professor. - Nobel x2 (Physics + Chemistry); first woman Nobel laureate; only person with Nobels in two sciences. - Discovered polonium + radium; coined "radioactivity"; proved it is an atomic property. ## More - birthplace, death place, age, cause of death - degree years, in-school firsts x2 - element naming origin, collaborators, full timeline - Nobel year per prize, co-laureates, citation - device name, deployment scale, patients treated - notebook radioactivity, storage, access conditions ``` Now begin. ```` ### User prompt template ```text Downstream model will read your compressed block to decide whether to expand it. Compress faithfully: preserve the passage topic + core facts. Do NOT invent facts. Do NOT drop major facts. Do NOT write meta-commentary about the Query (never write "Query info: absent", "no X mention", etc.); if the passage does not address the Query, still summarize the passage. ## Query (ordering hint only — still summarize the whole passage) {query} ## Target length Compress AS MUCH AS faithfully possible. HARD CEILING: {budget} chars (~50% of the source). If core facts fit in far fewer chars, output fewer. Never exceed the ceiling. ## Passage {title}: {body} ``` - `{query}` — the user question providing an ordering hint (not a filter). - `{budget}` — hard character ceiling; recipe: `max(160, int(len(title+body) * 0.5))`. - `{title}: {body}` — passage title prefixed to the body. ### Assistant output format ```text ## Summary ## More ``` --- ## 2. Training Recipe Reproduce with: ```bash torchrun --nproc_per_node=8 cookbook/rl/train_condenser_ddp.py ``` ### 2.1 Setup | Item | Value | |---|---| | Base model | `Qwen/Qwen3.5-4B` (loaded via `ms://Qwen/Qwen3.5-4B`) | | Template | `Qwen3_5Template`, `max_length=4096` | | Dataset | [`twinkle-kit/hotpotqa-condensed-9k`](https://www.modelscope.cn/datasets/twinkle-kit/hotpotqa-condensed-9k) (saved locally as `ds_condensed.jsonl`) | | Parallelism | **Pure DDP** — `DeviceMesh.from_sizes(dp_size=8)`, 8 × GPU | ### 2.2 LoRA | Item | Value | |---|---| | Rank `r` | **16** | | `lora_alpha` | 32 | | `target_modules` | `all-linear` | | Adapter name | `default` | ### 2.3 Optimization | Item | Value | |---|---| | Optimizer | AdamW | | Learning rate | `1e-4` | | Batch size | 8 (per DP rank) | | Gradient accumulation | 4 | | LR scheduler | `CosineWarmupScheduler`, 50 warmup steps, `len(dataloader) * NUM_EPOCHS` total | | Epochs | 5 | ### 2.4 Notes on the DDP + LoRA contract - `model.model._no_split_modules = {'Qwen3_5DecoderLayer'}` keeps each decoder layer atomic during any later sharding/activation-checkpoint pass. --- ## 3. Inference Minimum viable local call (vLLM or any OpenAI-compatible server exposing the LoRA). Fill `SYSTEM` with the **System prompt** block above and `USER_TEMPLATE` with the **User prompt template** block above. ```python from openai import OpenAI client = OpenAI(base_url='http://localhost:8000/v1', api_key='EMPTY') SYSTEM = '...' # paste the System prompt block verbatim USER_TEMPLATE = '...' # paste the User prompt template block, keeping {query}/{budget}/{text} query = 'Which city was Marie Curie born in?' title = 'Marie Curie' body = '' text = f'{title}: {body}' budget = max(160, int(len(text) * 0.5)) resp = client.chat.completions.create( model='qwen3.5-4b-condenser', messages=[ {'role': 'system', 'content': SYSTEM}, {'role': 'user', 'content': USER_TEMPLATE.format( query=query, budget=budget, text=text)}, ], temperature=0.3, max_tokens=max(128, int(budget * 0.6) + 16), ) print(resp.choices[0].message.content) ``` For on-trace usage inside the Twinkle compression pipeline, the model is plugged in via `ModelCondenser` as the sampler backend — see `src/twinkle_agentic/condenser/model.py`. --- ## 4. Expected Behaviour - **Format compliance**: `## Summary` + `## More` sections, telegraphic, no meta-commentary. SFT convergence is clean enough that format errors are rare. - **Ratio**: typically **0.35–0.55** of source length; hard ceiling `budget * 1.15` (the data-side acceptance gate). - **Query-as-ordering-hint**: facts that hit the Query are front-loaded in `## Summary`; other core facts are still retained either inline (if budget allows) or indexed in `## More`. - **`More` as an index**: category keywords only — `member roles`, `prior bands`, `album title`, etc. Values should NOT be inlined in `More`. --- ## 5. Known Limitations - **Short-passage long tail**: on sources below ~250 chars, the achieved ratio can drift above 0.7 because there is little to compress; mirrors the teacher-data long tail. - **Entity-attribute coverage**: low-frequency attributes (e.g. nationality like "English-American") may be dropped entirely rather than surfacing under `## More`. Scaling up data or adding targeted prompts helps. - **Language**: English only. Multilingual behaviour is untested. - **Not a standalone QA model**: the condenser only rewrites a passage. Downstream answering is the job of the agent that decides when to call `extract_condensed` on a compressed block. --- ## 6. Intended Use - Drop-in compression backend for the Twinkle Agentic RL pipeline (`cookbook/rl/grpo_condensed.py`). - SFT warm-start for subsequent GRPO / DPO against the same condenser schema. - Offline dataset preprocessing: pre-compress long corpora to control context budgets at retrieval time. --- ## 7. Download :modelscope-code[]{type="sdk"} :modelscope-code[]{type="git"}