---
title: Qwen-AgentWorld-35B-A3B-GGUF
canonical_url: "https://www.modelscope.cn/models/groxaxo/Qwen-AgentWorld-35B-A3B-GGUF"
md_url: "https://www.modelscope.cn/models/groxaxo/Qwen-AgentWorld-35B-A3B-GGUF.md"
repository: groxaxo/Qwen-AgentWorld-35B-A3B-GGUF
last_updated: 2026-08-22
base_model:
  - Qwen/Qwen-AgentWorld-35B-A3B
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
downloads: 142
stars: 1
tags:
  - Qwen
  - agent
  - world
  - 35b
  - a3b
  - moe
  - gguf
---

# Qwen-AgentWorld-35B-A3B-GGUF

> Qwen-AgentWorld-35B-A3B-GGUF - groxaxo 在 ModelScope 开源的模型。Qwen-AgentWorld-35B-A3B GGUF

groxaxo/Qwen-AgentWorld-35B-A3B-GGUF 是 ModelScope 魔搭社区上的机器学习模型，基于 Qwen/Qwen-AgentWorld-35B-A3B 构建。

- **Repository**: groxaxo/Qwen-AgentWorld-35B-A3B-GGUF
- **Base model**: Qwen/Qwen-AgentWorld-35B-A3B
- **Tags**: Qwen, agent, world, 35b, a3b, moe, gguf
- **Downloads**: 142
- **Stars**: 1
- **Last updated**: 2026-08-22

Source: https://www.modelscope.cn/models/groxaxo/Qwen-AgentWorld-35B-A3B-GGUF

---

# Qwen-AgentWorld-35B-A3B GGUF

<!-- polished-overview:start -->
## Overview

**Qwen-AgentWorld-35B-A3B-GGUF** is a GGUF release for llama.cpp-compatible runtimes and local inference, published by [`groxaxo`](https://huggingface.co/groxaxo).
It is intended for open-source evaluation, reproducible experimentation, and compatible local or
hosted inference workflows. The wording below is deliberately limited to what can be verified
from this repository's metadata and artifacts.


## At a glance

| Field | Details |
|---|---|
| Format | **GGUF** |
| Source / base | [`Qwen/Qwen-AgentWorld-35B-A3B`](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B) |
| Intended task | the task described by the included configuration and documentation |
| License | the license declared in the repository files |

## What is included

- `*.gguf` (6 files)
- Additional configuration, tokenizer, processor, or shard files (6 visible artifacts total)

## Quick start

### llama.cpp

Download a `.gguf` file that fits your available memory, then run it with a current llama.cpp
build:

```bash
llama-cli \
  -m /path/to/model.gguf \
  -p "Write a concise technical summary."
```

For vision or any-to-any models, download the matching multimodal projection file when one is
provided and follow the source model's modality-specific instructions.

## Compatibility and responsible use

- Use a runtime that explicitly supports this format, architecture, and modality.
- Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
- Review the source model card and license before redistribution or deployment.
- Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
- Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for
testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.
<!-- polished-overview:end -->

GGUF quantizations of [Qwen/Qwen-AgentWorld-35B-A3B](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B) generated with [llama.cpp](https://github.com/ggerganov/llama.cpp).

- Architecture: `qwen35moe` (35B params, 256 experts / 8 active, A3B)
- Context length: 262 144
- Source: BF16 GGUF converted via `convert_hf_to_gguf.py`
- Importance matrix (`coding.imatrix.gguf`) generated from a curated coding calibration text (1 840 samples), 128 chunks, ctx 4 096.

## Files

| File | Size (GiB) | BPW | Use |
| --- | --- | --- | --- |
| `Qwen-AgentWorld-35B-A3B-BF16.gguf` | 64.61 | 16.01 | reference / highest fidelity |
| `Qwen-AgentWorld-35B-A3B-Q8_0.gguf` | 34.37 | 8.52 | near-lossless, needs >24 GB VRAM with all layers offloaded |
| `Qwen-AgentWorld-35B-A3B-Q6_K.gguf` | 26.56 | 6.58 | very high quality, ~20 GB VRAM |
| `Qwen-AgentWorld-35B-A3B-Q4_K_M.gguf` | 19.71 | 4.88 | balanced quality / size, ~14 GB VRAM |
| `Qwen-AgentWorld-35B-A3B-Q2_K.gguf` | 12.05 | 2.99 | smallest, ~9 GB VRAM, quality trade-off |
| `Qwen-AgentWorld-35B-A3B-coding.imatrix.gguf` | 0.18 | — | importance matrix for finer quant recipes (Tensor-type overrides). |

Plain quantization passes (no recipe overrides) only — these avoid the
`std::bad_alloc` triggered by `--tensor-type-file` + imatrix on this
specific MoE architecture in the current llama.cpp build (8194 /
1179bfc82). The imatrix file is still provided for users who want to
mix types via `--tensor-type-file`.

## Recommended inference

```
llama.cpp/build/bin/llama-server \
    -m Qwen-AgentWorld-35B-A3B-Q4_K_M.gguf \
    -ngl 999 --ctx-size 8192 -b 2048 -ub 512 -np 1 \
    --temp 0.6 --top-p 0.95 --top-k 20 \
    -fa --host 0.0.0.0 --port 8080
```