---
title: Diplodocus-preview-GGUF
canonical_url: "https://www.modelscope.cn/models/OsaurusAI/Diplodocus-preview-GGUF"
md_url: "https://www.modelscope.cn/models/OsaurusAI/Diplodocus-preview-GGUF.md"
repository: OsaurusAI/Diplodocus-preview-GGUF
last_updated: 2026-10-15
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
base_model:
  - IFM/K2-Horizon-7B
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 12
stars: 0
tags:
  - gguf
  - llama.cpp
  - quantized
  - imatrix
  - reasoning
  - tool-use
  - finance
  - documents
  - spreadsheets
  - k2_horizon
---

# Diplodocus-preview-GGUF

> Diplodocus-preview-GGUF - OsaurusAI 在 ModelScope 开源的模型。OsaurusAI/Diplodocus-preview-GGUF

OsaurusAI/Diplodocus-preview-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 IFM/K2-Horizon-7B 构建。

- **Repository**: OsaurusAI/Diplodocus-preview-GGUF
- **License**: apache-2.0
- **Tasks**: text-generation
- **Base model**: IFM/K2-Horizon-7B
- **Tags**: gguf, llama.cpp, quantized, imatrix, reasoning, tool-use, finance, documents, spreadsheets, k2_horizon
- **Downloads**: 12
- **Stars**: 0
- **Last updated**: 2026-10-15

Source: https://www.modelscope.cn/models/OsaurusAI/Diplodocus-preview-GGUF

---

<p align="center">
  <a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a>
</p>

# OsaurusAI/Diplodocus-preview-GGUF

**Diplodocus preview 4**, the Osaurus assistant for finance, documents, spreadsheets and tool-assisted work, built on IFM/K2-Horizon-7B and trained on Osaurus' own workspace tools (workbooks and documents edited in place, the built-in `calculate` tool, reasoning on and off). This is the **GGUF build for llama.cpp** in three calibrated sizes:

| File | Size | Use |
|---|---|---|
| `Diplodocus-preview-Q4_K_M.gguf` | 6.33 GB | smallest; 8 GB GPUs / tight RAM |
| `Diplodocus-preview-Q6_K.gguf` | 7.76 GB | **recommended** balance |
| `Diplodocus-preview-Q8_0.gguf` | 9.57 GB | closest to BF16 |

Using Osaurus on a Mac? Use the MLX bundle [`OsaurusAI/Diplodocus-preview-JANG_6M`](https://huggingface.co/OsaurusAI/Diplodocus-preview-JANG_6M) instead. Same weights, same template, same reasoning toggle.

**Requires llama.cpp with K2 Horizon support** ([ggml-org/llama.cpp#29535](https://github.com/ggml-org/llama.cpp/pull/29535), merged 2026-10-06). Older builds, and apps that bundle an older llama.cpp, will not load the `k2-horizon` architecture.

## What preview 4 adds over preview 3

Trained rounds R8–R11 on top of preview 3: the Osaurus `calculate` tool (when to call it, chained money arithmetic, recovery from engine errors), workbooks and documents through Osaurus' in-place `file_edit` operations (bounded `file_read` windows, `mode:"structure"`, `set_cells` with live formulas, never regenerating an existing workbook), PDF forms and signed-PDF handling, and a loop fix for reasoning-off answers. The source's own held-out evaluation moved end-to-end workbook/document tasks from 59/51 (on/off, of 100) to 70/70 and calculator-executed long-document money answers from 31 to 43 of 60 with reasoning on.

## Quantization

| Component | Q4_K_M | Q6_K | Q8_0 |
|---|---|---|---|
| Attention projections (q, k, v, o; all 36 layers) | **Q8_0** | **Q8_0** | Q8_0 |
| MLP gate / up (all 36 layers) | Q4_K, importance-weighted | Q6_K, importance-weighted | Q8_0 |
| MLP down (all 36 layers) | Q6_K in 18 layers, Q4_K in 18, importance-weighted | Q6_K, importance-weighted | Q8_0 |
| Embeddings | Q4_K | Q6_K | Q8_0 |
| Untied output head | Q6_K | Q6_K | Q8_0 |
| Norms | F32 | F32 | F32 |

| File | Bytes | sha256 |
|---|---|---|
| `Diplodocus-preview-Q4_K_M.gguf` | 6,327,738,208 | `32c4d81cacb4af48c61e852e8bca2d974548900aca337088c14c2869e15311e4` |
| `Diplodocus-preview-Q6_K.gguf` | 7,760,248,672 | `c81c80cc2f3d5f9fec0eb06d9a8732404ac37516fde2d8692eb49f6e1d39cfd9` |
| `Diplodocus-preview-Q8_0.gguf` | 9,573,973,632 | `8d39a2d47b56b308d60d79efa194a2f0796571b439b14499cd4847ccacdf8a16` |

The importance matrix (`Diplodocus-preview.imatrix.gguf`, included) was computed on the BF16 model over **1,653,067 tokens** of Osaurus tool conversations, reasoning-on/off traces and calculator chains, with chat and tool special tokens parsed as the model sees them.

The layouts were chosen by measurement, not by default. Every candidate was scored against the BF16 source on a held-out set disjoint from calibration: 64 rows, 206k tokens, reasoning on and off, 41 with tool calls. Lower KL divergence is better; "top-1 same" is how often the quant picks the same next token as BF16.

| Candidate | Size | Mean KLD | Median KLD | Top-1 same |
|---|---|---|---|---|
| **Q8_0 (shipped)** | **9.57 GB** | **0.0285** | **0.0019** | **95.5%** |
| Q6_K, standard | 7.39 GB | 0.1325 | 0.0109 | 89.7% |
| Q6_K + importance matrix | 7.39 GB | 0.1020 | 0.0076 | 91.3% |
| **Q6_K + importance matrix + Q8_0 attention (shipped)** | **7.76 GB** | **0.0608** | **0.0044** | **93.1%** |
| Q6_K shipped + Q8_0 output head | 8.01 GB | 0.0603 | 0.0039 | 93.5% |
| Q4_K_M + importance matrix | 5.59 GB | 0.2055 | 0.0311 | 85.3% |
| **Q4_K_M + importance matrix + Q8_0 attention (shipped)** | **6.33 GB** | **0.1246** | **0.0182** | **88.4%** |
| Q4_K_M shipped + Q6_K MLP down | 6.56 GB | 0.1110 | 0.0149 | 89.3% |

At both Q6 and Q4, moving the attention projections to 8-bit removed roughly 40% of the divergence for +0.37 / +0.74 GB. This matches the JANG bundle's finding that preview 4's fine-tuning lives mostly in attention. The next steps up (8-bit output head for Q6, 6-bit MLP down for Q4) bought much less per byte and would crowd the next size tier, so they were not shipped. For scale, this model is noise-sensitive: the JANG card measured BF16 with one-ULP embedding noise at 96.2% top-1 on its set, and Q8_0 sits at 95.5% here. These are quantization diagnostics on training-distribution data, not an independent task-accuracy benchmark.

## Reasoning, tools and sampling

- **High:** reasoning on (`reasoning_effort: "high"`, the default). **Low:** reasoning off, using the embedded template's empty prefilled reasoning block. Do not send `medium`.
- The embedded chat template is the same template shipped with the JANG bundle. Run `llama-server` with `--jinja` so it is used.
- Default sampling is **temperature 1.0, top_p 0.95**, stamped in the GGUF metadata.
- Tool calls use the native `k2_horizon` envelope. Upstream `llama-server --jinja` parses them into OpenAI `tool_calls` and puts reasoning in `reasoning_content`.
- For money and exact arithmetic, use reasoning on and let the model drive the `calculate` tool.

```bash
llama-server -hf OsaurusAI/Diplodocus-preview-GGUF:Q6_K --jinja -ngl 99 -c 32768   # or :Q4_K_M / :Q8_0
```

```bash
curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is 1234.56 * 0.0825, rounded to cents?"}],
  "tools": [{"type": "function", "function": {"name": "calculate", "description": "Evaluate an arithmetic expression exactly.",
             "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}}],
  "chat_template_kwargs": {"reasoning_effort": "high"}
}'
```

Send `"chat_template_kwargs": {"reasoning_effort": "low"}` to turn reasoning off.

## Validation

Every shipped file was checked in upstream `llama-server` (llama.cpp `a0e6dc738`) on an NVIDIA GB10:

- **Template and tokens:** the rendered prompt carries exactly one BOS; reasoning-off prompts end in the empty prefilled block. Generation stops at `<|ifm|im_end|>`; `<|ifm|endoftext|>` also stops.
- **Reasoning tags:** reasoning-off outputs contain no reasoning tags; reasoning-on outputs close with exactly one `</ifm|think>`.
- **OpenAI API:** reasoning on returns `reasoning_content` plus clean content; reasoning off returns clean content with no reasoning. A `calculate` request returns `finish_reason: tool_calls` with parsed arguments, and after the tool result the final answer is the correct **$101.85**.
- **Tool use under the default sampling** (temperature 1.0, top_p 0.95, reasoning on, 40 runs over four money prompts; think-tag integrity 40/40 for every file):

| | BF16 | Q8_0 | Q6_K | Q4_K_M |
|---|---|---|---|---|
| Calculator called | 39/40 | 38/40 | 38/40 | 40/40 |

The differences are within sampling noise.

## Known limits

From the source's own evaluation: new workbooks with live formulas sometimes end up with text formulas; change requests on signed PDFs with reasoning on are handled in 3 of 10 cases; money arithmetic with reasoning off is unreliable, so use reasoning on (in our checks, reasoning-off answered a calculator question from memory, and both BF16 and Q6_K got it wrong). The model may identify itself as K2 and may attempt a tool from older conversation history. Use current tool definitions and validate consequential outputs.

## 한국어 안내

Diplodocus 프리뷰 4의 llama.cpp용 GGUF 모델입니다. Q4_K_M(6.33 GB), Q6_K(7.76 GB, 권장), Q8_0(9.57 GB) 세 가지로 제공되며, Q4_K_M과 Q6_K는 어텐션을 Q8_0으로 두고 나머지를 중요도 행렬로 보정했습니다. K2 Horizon을 지원하는 llama.cpp(#29535 이후)가 필요하며 `--jinja`로 실행하세요. 추론은 `reasoning_effort`를 `high`로 켜거나 `low`로 끌 수 있고, 기본 샘플링은 temperature 1.0, top_p 0.95입니다. 금액 계산에는 추론을 켜고 `calculate` 도구를 사용하세요. Mac의 Osaurus에서는 [`Diplodocus-preview-JANG_6M`](https://huggingface.co/OsaurusAI/Diplodocus-preview-JANG_6M)을 사용하세요.

Quantized and validated by **Jinho Jang** — eric@osaurus.ai