--- title: NeoHorse-1-9B-GGUF canonical_url: "https://www.modelscope.cn/models/TokenRhythm/NeoHorse-1-9B-GGUF" md_url: "https://www.modelscope.cn/models/TokenRhythm/NeoHorse-1-9B-GGUF.md" repository: TokenRhythm/NeoHorse-1-9B-GGUF last_updated: 2026-09-09 license: apache-2.0 pipeline_tag: text-generation tasks: - text-generation base_model: - TokenRhythm/NeoHorse-1-9B base_model_relation: quantized library_name: - gguf downloads: 335 stars: 2 tags: - gguf - llama-cpp - agentic - tool-use - coding - reasoning - instruction-following --- # NeoHorse-1-9B-GGUF > NeoHorse-1-9B-GGUF - TokenRhythm 在 ModelScope 开源的模型。NeoHorse-1-9B GGUF: BF16, F16, Q8_0, Q5_K_M, Q4_K_M. TokenRhythm/NeoHorse-1-9B-GGUF 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可，基于 TokenRhythm/NeoHorse-1-9B 构建。 - **Repository**: TokenRhythm/NeoHorse-1-9B-GGUF - **License**: apache-2.0 - **Tasks**: text-generation - **Base model**: TokenRhythm/NeoHorse-1-9B - **Tags**: gguf, llama-cpp, agentic, tool-use, coding, reasoning, instruction-following - **Downloads**: 335 - **Stars**: 2 - **Last updated**: 2026-09-09 Source: https://www.modelscope.cn/models/TokenRhythm/NeoHorse-1-9B-GGUF ---

NeoHorse-1-9B-GGUF

Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness.

Technical Report

NeoHorse-1-9B is a 9B causal language model and an initial prototype on the path toward **recursive self-improvement (RSI)**. It is post-trained from Qwen3.5-9B for text-based agent harnesses, tool use, coding, and instruction following. This repository provides **GGUF versions** of [NeoHorse-1-9B](https://huggingface.co/TokenRhythm/NeoHorse-1-9B) for local use with llama.cpp, Ollama, and LM Studio. It includes **16-bit (BF16)** weights and smaller **8-bit, 5-bit, and 4-bit quantized versions**. Quantized versions take up less disk space and use less memory, making the model easier to run on your own hardware. These files contain **text-only model weights**, fine-tuned by TokenRhythm from [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B).

## Highlights - **Path toward RSI:** the routing harness assigns tasks to a heterogeneous model pool, records tool interactions and outcomes, estimates capability demand, and uses capability-level feedback to shape the next training mixture. Updated models can return to the harness, closing a prototype evaluation–selection–update loop; extending this loop across successive iterations is the next step toward RSI. - **Agentic post-training framework:** the associated research explores routing-guided curriculum SFT and routing-guided on-policy distillation to turn execution trajectories into training signal while preserving execution and harness context around each response. - **Data quality:** exact and near-duplicate removal, evaluation decontamination, structural validation, six-dimensional semantic evaluation, and subscene-level Scene/Goal/Outcome labeling. - **Broad gains:** 69.04 macro average across ten benchmarks versus 65.60 for Qwen3.5-9B (**+3.44**). ## Model Details

Property	Value
Model family	NeoHorse Agent-Native Causal Language Model
Parameters	Approximately 9B
Base model	Qwen3.5-9B
Post-training	Routing-guided agentic post-training
Interface	Text input and text output
Context length	262,144 natively and extensible up to 1,010,000 tokens.
Weight format / precision	GGUF / 16-bit (BF16), 8-bit, 5-bit, 4-bit

## Evaluation The results below are from the original checkpoint, not a separate GGUF benchmark run. The 9B track compares NeoHorse-1-9B with five representative open-weight baselines: Granite-4.2-8B, Qwen3.5-9B, Ornith-1.5-9B, Gemma-4-12B-it, and Muse-Glimmer-30B. Results cover ten benchmarks and are grouped by capability. Higher is better; `Δ` is NeoHorse-1-9B minus Qwen3.5-9B. **Bold** and underline mark the best and second-best results in each benchmark row, respectively; ties share the same formatting.

Benchmark	Granite-4.2-8B	Qwen3.5-9B	Ornith-1.5-9B	Gemma-4-12B-it	Muse-Glimmer-30B	NeoHorse-1-9B	Δ vs Qwen3.5-9B
🤖 Agentic
QwenClawBench	37.01	44.04	47.27	43.53	46.11	48.73	+4.69
WorkBuddy Bench	35.07	39.60	29.29	29.65	45.85	40.15	+0.55
PinchBench	56.93	74.55	68.22	58.89	71.35	82.25	+7.70
VitaBench	23.00	31.25	26.75	36.50	48.50	42.25	+11.00
BFCL v4	52.06	64.88	65.03	62.06	53.74	67.43	+2.55
tau2-Bench	62.28	88.04	83.68	59.37	76.64	90.82	+2.78
💻 Coding
HumanEval	96.34	92.68	93.90	100.00	98.17	98.17	+5.49
LiveCodeBench v6	72.00	65.14	47.43	73.14	65.71	65.14	+0.00
📚 Instruction Following
IFBench	78.00	66.33	40.00	77.67	78.67	66.33	+0.00
IFEval	92.98	89.46	71.35	94.27	93.90	89.09	-0.37
📊 Overall
Ten-benchmark average	60.57	65.60	57.29	63.51	67.86	69.04	+3.44

> **Reported protocol:** SGLang v0.5.17 · `temperature=1.0` · `top_p=0.95` · `top_k=20` · `min_p=0.0` · `presence_penalty=1.5` · `repetition_penalty=1.0` · thinking mode enabled with `enable_thinking=true` and `force_nonempty_content=true`. QwenClawBench, WorkBuddy Bench, and tau2-Bench use three runs; PinchBench and VitaBench use one run; the remaining benchmarks follow their official protocols. VitaBench uses the DeepSeek-V4-Flash simulator and judge. ## Deployment **FP16 option:** `NeoHorse-1-9B-F16.gguf` (17.92 GB) is also available for runtimes or hardware that prefer FP16 over BF16. Use `:F16` instead of `:BF16` in the llama.cpp or Ollama commands below. This is a format-converted 16-bit option, not a quality upgrade. ### Quantized files | Version | File | Size | |---|---|---:| | 8-bit | `NeoHorse-1-9B-Q8_0.gguf` | 9.53 GB | | 5-bit | `NeoHorse-1-9B-Q5_K_M.gguf` | 6.47 GB | | 4-bit | `NeoHorse-1-9B-Q4_K_M.gguf` | 5.63 GB | **4-bit** is the smallest option, **5-bit** sits in the middle, and **8-bit** uses less compression. Choose **16-bit (BF16)** to keep the original weight precision. Lower precision saves memory but can affect response quality. The sizes above are file sizes; running the model also needs memory for the conversation and other runtime data. These standard llama.cpp quantizations were generated directly from the BF16 GGUF, without an importance matrix. All three passed eight short llama.cpp compatibility checks, including thinking, single and parallel tool calls, and tool-result continuation, at 4K context. These checks are not a quantized-quality benchmark; the benchmark scores in this card refer to the original checkpoint. Quantized variants have not yet been separately tested in Ollama or LM Studio. Use the desired filename in place of the BF16 filename in the examples below. Use a recent runtime with Qwen3.5 support. The commands below download the model from Hugging Face and cache it locally; no manual file download is required. These are text-only GGUF files with an embedded chat template and no MTP draft head. ### llama.cpp Download and start a local OpenAI-compatible server: ```bash llama-server -hf TokenRhythm/NeoHorse-1-9B-GGUF:BF16 -c 4096 -ngl 99 --jinja --port 8080 --alias neohorse-1-9b ``` ### Ollama Download and run directly, without creating a Modelfile: ```bash ollama run hf.co/TokenRhythm/NeoHorse-1-9B-GGUF:BF16 ``` ### LM Studio Paste [this repository URL](https://huggingface.co/TokenRhythm/NeoHorse-1-9B-GGUF) into LM Studio's model search and choose a GGUF file, or download through the CLI: ```bash lms get https://huggingface.co/TokenRhythm/NeoHorse-1-9B-GGUF ``` Select the desired file when prompted, then load it in LM Studio with MTP speculative decoding disabled. Start with a 4K context and increase it according to available memory. In LM Studio's native API, use `reasoning: "off"` to disable thinking. For llama.cpp and Ollama, replace `:BF16` with the tag of another variant listed in **Files and versions**, such as `:Q8_0`, `:Q5_K_M`, or `:Q4_K_M`. Only variants already present in the repository can be downloaded. See the [llama.cpp guide](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md), [Ollama–Hugging Face integration](https://huggingface.co/docs/hub/ollama), and [LM Studio download guide](https://lmstudio.ai/docs/cli/local-models/get).

Manual download and local-file setup (optional)

Download `NeoHorse-1-9B-BF16.gguf` (approximately 17.92 GB). Use a recent runtime with Qwen3.5 support. The file includes the chat template and does not include an MTP draft head. ### llama.cpp ```bash llama-server -m NeoHorse-1-9B-BF16.gguf -c 4096 -ngl 99 --jinja --port 8080 --alias neohorse-1-9b ``` ### Ollama Create a `Modelfile` containing: ```text FROM ./NeoHorse-1-9B-BF16.gguf PARAMETER num_ctx 4096 ``` ```bash ollama create neohorse-1-9b-bf16 -f Modelfile ollama run neohorse-1-9b-bf16 ``` ### LM Studio Import the GGUF file and load it with MTP speculative decoding disabled. With the CLI: ```bash lms import ./NeoHorse-1-9B-BF16.gguf --copy lms load neohorse-1-9b --context-length 4096 --no-speculative-draft-mtp ``` Basic chat and tool-call smoke tests passed in llama.cpp, Ollama 0.33.3, and LM Studio's llama.cpp 2.33.0 engine. Context capacity depends on available memory; the examples start at 4K. In LM Studio's native API, use `reasoning: "off"` to disable thinking.

## Using NeoHorse-1-9B-GGUF via the Chat Completions API After starting `llama-server` as shown in [Deployment](#deployment), connect an OpenAI-compatible client to `http://127.0.0.1:8080/v1`. The examples below use the server alias `neohorse-1-9b`; it must match the `--alias` value in the launch command. ### Basic Usage Install the Python client: ```bash pip install openai ``` Send a message to your local server: ```python from openai import OpenAI client = OpenAI( base_url="http://127.0.0.1:8080/v1", api_key="EMPTY", # Replace if server authentication is enabled. ) generation_config = { "temperature": 1.0, "top_p": 0.95, "presence_penalty": 1.5, "max_tokens": 2048, "extra_body": { "top_k": 20, "min_p": 0.0, "repeat_penalty": 1.0, "chat_template_kwargs": {"enable_thinking": True}, }, } response = client.chat.completions.create( model="neohorse-1-9b", messages=[ { "role": "user", "content": "Write a Python function that returns the first n Fibonacci numbers.", } ], **generation_config, ) message = response.choices[0].message print("Reasoning:", getattr(message, "reasoning_content", None)) print("Answer:", message.content) ``` These short examples allow up to 2,048 output tokens within the 4K server context. For longer prompts or responses, increase `-c` and adjust `max_tokens` while leaving room for the input and chat template. ### Tool Calling Keep `--jinja` enabled in the `llama-server` launch command so it can use the embedded tool-aware chat template and parse generated function calls into `tool_calls`. Using the same `client` and `generation_config` from Basic Usage: ```python tools = [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather for a specified city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "Name of the city.", } }, "required": ["city"], }, }, } ] response = client.chat.completions.create( model="neohorse-1-9b", messages=[ { "role": "user", "content": "What is the weather in Beijing right now?", } ], tools=tools, tool_choice="auto", **generation_config, ) message = response.choices[0].message if message.tool_calls: for call in message.tool_calls: print(call.function.name, call.function.arguments) else: print(message.content) ``` This example prints the requested function and its arguments. Your application executes the function and returns its result in a `tool` message, using the corresponding `tool_call_id`, to continue the conversation. ## License NeoHorse-1-9B is released under the **Apache License 2.0**. The upstream model is [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B). Its original copyright notice, Copyright 2026 Alibaba Cloud, is retained in the license file. TokenRhythm has modified the model through fine-tuning and repackaging for text-only inference. This repository converts the resulting checkpoint to BF16 GGUF and provides quantized variants using llama.cpp. ## Citation ``` @misc{neohorse2026, title = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness}, author = {NeoHorse Team}, year = {2026}, howpublished = {arXiv preprint}, eprint = {2609.08183}, archivePrefix = {arXiv}, primaryClass = {cs.CL}, url = {https://arxiv.org/abs/2609.08183} } ``` For questions or issue reports, use the [NeoHorse project repository](https://github.com/TokenRhythm/NeoHorse).