---
title: laya-MNN-fp16
canonical_url: "https://www.modelscope.cn/models/yunfeng/laya-MNN-fp16"
md_url: "https://www.modelscope.cn/models/yunfeng/laya-MNN-fp16.md"
repository: yunfeng/laya-MNN-fp16
last_updated: 2026-09-23
license: apache-2.0
pipeline_tag: text-classification
tasks:
  - text-classification
downloads: 0
stars: 0
tags:
  - mnn
  - onnx
  - laya
  - system-one
  - decision-model
  - calibrated-decisions
  - edge-inference
  - quantized
---

# laya-MNN-fp16

> laya-MNN-fp16 - yunfeng 在 ModelScope 开源的模型。MNN build of Laya — a multilingual, non-autoregressive System 1 decision model. Give it a state (email, ticket, JSON) and typed questions; it returns typed answers with calibrated probabilities in a single forward…

yunfeng/laya-MNN-fp16 是 ModelScope 魔搭社区上的text-classification模型，采用 apache-2.0 许可。

- **Repository**: yunfeng/laya-MNN-fp16
- **License**: apache-2.0
- **Tasks**: text-classification
- **Tags**: mnn, onnx, laya, system-one, decision-model, calibrated-decisions, edge-inference, quantized
- **Downloads**: 0
- **Stars**: 0
- **Last updated**: 2026-09-23

Source: https://www.modelscope.cn/models/yunfeng/laya-MNN-fp16

---

# Laya for MNN

MNN build of [Laya](https://huggingface.co/convaiinnovations/laya) — a multilingual, non-autoregressive System 1 decision model. Give it a **state** (email, ticket, JSON) and **typed questions**; it returns typed answers with calibrated probabilities in a single forward pass. It never generates text, so there is nothing to parse and nothing to hallucinate.

This repository hosts `fp16` and `int8` checkpoints of the **English** model (`ModernBERT-large` backbone, 421M parameters), converted for the [MNN](https://github.com/alibaba/MNN) runtime so they run on CPU without PyTorch.

| repo | precision | size | verification vs. original PyTorch |
|---|---|---|---|
| [`laya-MNN-fp16`](https://huggingface.co/yunfengwang/laya-MNN-fp16) | fp16 weights | 846 MB | 9/9 answers match, worst probability delta **0.0019** |
| [`laya-MNN-int8`](https://huggingface.co/yunfengwang/laya-MNN-int8) | 8-bit weight-only | 581 MB | 9/9 argmax match, worst probability delta **0.0343** |

Use `fp16` unless size is critical.

## Files

```
laya-fp16.mnn     converted weights
tokenizer.json    ModernBERT BPE tokenizer (used directly, no transformers dependency)
manifest.json     config, sha256, max_len/head_max_len/max_options, verification results
```

## Run it

```bash
uvx laya-mnn decide \
  --state "Hi, we were billed twice for March. Please refund the duplicate or we will cancel our plan." \
  --questions '{"department":{"type":"choice","instructions":"Which department should handle this?","criteria":{"billing":"invoices, payments, refunds","technical":"bugs, outages","other":"everything else"}}}'
```

The CLI downloads these weights automatically (ModelScope first, then Hugging Face) into `~/.cache/laya-mnn`. Source: [laya-mnn on PyPI](https://pypi.org/project/laya-mnn/).

```python
from laya_mnn import LayaMNN, ensure_model
model = LayaMNN(ensure_model("fp16"), threads=4)
```

## Conversion

```
convaiinnovations/laya (PyTorch)
  -> ONNX opset 17, eager attention, static [1, 512] sequence, 64 marker slots
  -> MNN converter 3.6.1 (--fp16 / --weightQuantBits 8)
```

Exported with `attn_implementation="eager"` so the graph stays as plain MatMul/Softmax/LayerNorm/GELU rather than a fused Attention node, and with static shapes so constant folding resolves the RoPE tables and the sliding-window masks. The graph contains `GatherElements`, `Erf`, `TopK`, `Cos`, `Sin`, `Mod`, `LessOrEqual` and `Where`; all are supported by the MNN ONNX importer.

### Verification

Each checkpoint was compared against the original PyTorch model on three sample states covering all three primitives (`choice`, `score`, `noul`), 9 questions in total, using the same temperatures the checkpoint ships with:

| | fp16 | int8 |
|---|---|---|
| answers matching | 9/9 | 9/9 (argmax) |
| worst probability delta | 0.0019 | 0.0343 |
| worst confidence delta | 0.0010 | 0.0343 |

## Limits

- **English only.** This is the repo-root checkpoint; the multilingual `mmBERT-base` variant is not converted here.
- **512 tokens of context, max 64 options per question**, 16 markers worth of head budget shared with the state text as in the original (`head_max_len = 192`).
- **Batch 1.** The graph is exported for a single sequence; MNN's shape inference mishandles the padding mask at larger batches, so questions are answered one at a time.
- **CPU only.** The Metal backend returned all-zero logits for this graph in the tested MNN 3.6.1 wheel.
- Calibration figures in the original model card are for the PyTorch model. The upstream checkpoint ships over-confident and expects temperature refitting on your own data — that applies here too.

## License and attribution

Apache-2.0. Model, prompt format and calibration scheme by [Convai Innovations](https://github.com/NandhaKishorM/laya) (Apache-2.0). This is an independent community conversion and is not affiliated with or endorsed by them.