---
title: TagGenerator
canonical_url: "https://www.modelscope.cn/models/itpossible/TagGenerator"
md_url: "https://www.modelscope.cn/models/itpossible/TagGenerator.md"
repository: itpossible/TagGenerator
chinese_name: TagGenerator
last_updated: 2025-05-31
model_type:
  - qwen2
architectures:
  - Qwen2ForCausalLM
parameters: 494.0M
tensor_type:
  - BF16
library_name:
  - safetensors
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 56
stars: 1
---

# TagGenerator

> TagGenerator - itpossible 在 ModelScope 开源的模型。TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Tasks

itpossible/TagGenerator 是 ModelScope 魔搭社区上的 494.0M 参数机器学习模型，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: itpossible/TagGenerator
- **Parameters**: 494.0M
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 56
- **Stars**: 1
- **Last updated**: 2025-05-31

Source: https://www.modelscope.cn/models/itpossible/TagGenerator

---

<div align="center">
    <h1>
        TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Tasks
    </h1>
</div>


## 🎉 News
- [2025-5-16] Our paper has been accepted for publication in ACL 2025.

## Introduction
Model routing allocates queries to the suitable model, improving system performance while reducing costs. However, existing routing methods face practical limitations that hinder scalability in large-scale applications and struggle to keep up with the rapid growth of the large language model (LLM) ecosystem. To tackle these challenges, we propose TagRouter, a training-free model routing method designed to optimize the synergy among multiple LLMs for open-domain text generation tasks. Experimental results demonstrate that TagRouter outperforms 13 baseline methods, increasing the accept rate of system by 6.15% and reducing costs by 17.20%, achieving optimal cost efficiency. Our findings provides the LLM community with an efficient and scalable solution for model ensembling, offering users an evolvable "super model."<br>

TagRouter consists of three modules: TagGenerator, TagScorer, and TagDecider. The TagGenerator is trained to generate a set of tags for a given query. The generated tags can be used for routing queries to the most suitable model based on their respective capabilities.

<p align="center">
    <br>
    <img src="image/TagRouter.png" width="800"/>
    <br>
</p>

## Download
[HuggingFace](https://huggingface.co/itpossible/TagGenerator)<br>

[ModelScope](https://modelscope.cn/models/itpossible/TagGenerator)


## Inference
Below is an example of inference code using TagGenerator.
```python
import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

os.environ["CUDA_VISIBLE_DEVICES"] = "0"

model_path = "itpossible/TagGenerator"
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

prompt = """[System]
You are an expert text tag extractor. Your task is to identify key tags that readers should focus on while engaging with the user query below.

[User Query]
Rewrite the sentence so that it's in the present tense: She had worked at the company for the past 3 years.
"""

messages = [
    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(**model_inputs, max_new_tokens=512)
generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(response)
```