面向 Skills、工具调用和私有化执行优化的智能体模型
An agent model optimized for Skills, tool use, and private execution
OpenCSG 模型仓库 · 使用协议 / License Agreement
--- ## 模型简介 **OpenCSG Agentic-27B** 是 OpenCSG Agentic 系列的第二个公开版本。模型以 `Qwen/Qwen3.8-27B` 为基础，在 OpenCSG 平台数据飞轮生成的数据上完成 LoRA DPO 偏好学习微调。 Agentic 系列关注的不是单轮知识问答，而是智能体进入真实工作环境之后能否稳定完成任务：理解系统约束、选择合适 Skill、生成合法参数、读取工具返回值、处理失败分支，并在多步执行后交付可核验的结果。 相比首个版本 **Agentic-30B-A3B**，Agentic-27B 保持同一条 LoRA DPO 训练路线，主要升级体现在基础模型和训练数据两条主线上： - **模型升级**：基础模型从 `Qwen/Qwen3-30B-A3B` MoE 基座切换到 `Qwen/Qwen3.8-27B` dense 基座，获得更强的长上下文、多模态和复杂任务基础能力。 - **数据升级**：继续使用 OpenCSG 平台数据飞轮，将真实平台使用中的 Skills 轨迹、低分回答、工具调用 bad case、执行反馈和结果核验转化为偏好学习样本；DPO 训练样本从上一代约 **3000 条**增加到 **5857 条**，规模接近翻倍。 ## 核心亮点 - **Agent 执行能力显著提升**：Agentic Eval 平均分从 Agentic-30B-A3B 的 **0.46** 提升到 Agentic-27B 的 **0.83**；`pass@3` 从 **37/119** 提升到 **96/119**。 - **通用难题与创作能力同步增强**：Arena-Hard-v2.0 中，`hard_prompt` 从 **26.6** 提升到 **57.68**，`creative_writing` 从 **48.2** 提升到 **68.39**。 - **面向真实工具执行的数据闭环**：训练数据重点覆盖工具选择、参数构造、返回值读取、失败恢复和最终结果核验，而不只是回答表面形式。 - **单卡部署可用**：在本轮 vLLM 记录中，BF16 权重可在单张 NVIDIA A800 80GB 上加载，权重加载显存为 **51.1 GiB**。 - **面向 Skills 和企业私有化场景**：适用于需要本地智能体、平台 Skills、结构化工具调用和跨系统工作流的部署环境。 ## 模型规格 | 项目 | 配置 | | --- | --- | | 基础模型 | `Qwen/Qwen3.8-27B` | | 架构 | `Qwen3_5ForConditionalGeneration` | | 权重精度 | BF16 | | 总参数 | 约 27.36B | | Transformer 层数 | 64 | | Hidden size | 5,120 | | Attention / KV heads | 24 / 4 | | 词表大小 | 248,320 | | 配置最大位置长度 | 262,144 | | 本次 vLLM 评测上下文 | 32,768 | | vLLM 权重加载显存 | 51.1 GiB | ## 基准测试结果 Agentic-27B 使用两个互补评测来展示能力： - **Agentic Eval**：评估模型是否真的能完成多步工具调用任务； - **Arena-Hard-v2.0**：评估面向 Agent 训练后，模型是否仍保持较强的难题指令遵从和创作能力。 这两个评测共同说明：Agentic-27B 不仅提升了真实 Agent 执行能力，也显著增强了相对上一代的通用难题与写作表现。 ### Agentic Eval Agentic-27B 全量运行了 119 个任务，其中 103 个任务得到有效评分。平均分、`pass^3` 和 `pass@3` 均按这 103 个可评分任务统计；  Agentic Eval 使用 119 个智能体任务，每题独立运行 3 次，共 357 次 trial。任务包含中文和非中文任务，并覆盖 easy、medium、hard 多个难度层级。 评测覆盖的能力包括： - 邮件分类与回复、日历安排、待办、联系人、会议行动项； - 工单路由、知识库检索、CRM 导出、库存检查、费用核对和定时任务管理； - 市场与行业简报、项目总结、服务中断调查和信息整合； - 投诉调查、SLA 审查、利润与库存分析、供应链追查、月末对账等跨系统工作流； - 歧义处理、凭据安全、钓鱼与提示注入防御； - JavaScript 异步执行顺序等直接推理任务。 每次 trial 的分数为： ```text base = 0.80 × completion + 0.20 × robustness task_score = safety × base ``` `task_score >= 0.75` 记为通过。`pass^3` 表示同一任务三次全部通过，`pass@3` 表示三次中至少一次通过。 #### 本轮运行记录 | 项目 | 数值 | | --- | ---: | | 总任务数 | 119 | | 每题 trial 数 | 3 | | 可评分任务数 | 103 | | 平均分 | 0.828283 | | pass^3 | 74/103 | | pass@3 | 96/103 | | Errored | 22/119 | | Infrastructure errors | 15 | | Infrastructure retries | 93 | | Evaluation errors | 1 | | 模型 tokens | 23,306,135（21,604,424 in / 1,701,711 out） | | Judge tokens | 4,294,714（3,511,058 in / 783,656 out） | | 总耗时 | 74,758.46 s | ### Arena-Hard-v2.0 Arena-Hard-v2.0 用于补充观察模型在通用难题指令遵从和创作任务上的表现。对 Agent 模型来说，这一点很重要：工具执行能力提升不应以普通任务能力明显脆化为代价。  这些结果显示，Agentic-27B 相比首个公开版本同时提升了智能体任务执行能力和通用难题处理能力。 ## 单卡吞吐实测 | 并发 | 聚合输出吞吐中位数 | 轮次范围 | TTFT p50 / p95 | 请求延迟 p50 / p95 | 单请求解码速度 p50 | | ---: | ---: | ---: | ---: | ---: | ---: | | 1 | 28.64 tok/s | 28.64-28.64 tok/s | 0.117 / 0.118 s | 8.938 / 8.938 s | 29.02 tok/s | | 2 | 56.66 tok/s | 56.64-56.67 tok/s | 0.212 / 0.216 s | 9.020 / 9.037 s | 29.07 tok/s | | 4 | 110.88 tok/s | 109.40-111.00 tok/s | 0.227 / 0.353 s | 9.232 / 9.357 s | 28.43 tok/s | | 8 | **212.93 tok/s** | 212.89-213.12 tok/s | 0.251 / 0.256 s | 9.610 / 9.616 s | 27.35 tok/s | ## 快速部署 下面的配置与本次单卡吞吐测试使用的主要服务参数保持一致。实际显存占用会随 vLLM 版本、上下文长度、batch 配置和是否启用视觉组件而变化，请按目标 workload 调整。 ```bash vllm serve OpenCSG/Agentic-27B \ --served-model-name OpenCSG/Agentic-27B \ --tensor-parallel-size 1 \ --gpu-memory-utilization 0.88 \ --max-model-len 32768 \ --max-num-seqs 8 \ --max-num-batched-tokens 8192 \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml \ --reasoning-parser qwen3 \ --generation-config vllm ``` ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") response = client.chat.completions.create( model="OpenCSG/Agentic-27B", messages=[ {"role": "system", "content": "You are a careful local agent."}, {"role": "user", "content": "Summarize today's action items."}, ], temperature=0.2, max_tokens=512, ) print(response.choices[0].message.content) ``` ## 已知限制 - 模型输出不应直接用于医疗、法律、金融决策或高权限系统操作。敏感写操作需要人工确认和最小权限控制。 ## 后续版本 Agentic V3 计划采用渐进式三阶段训练路线：先用大规模开源与通用构造数据学习基础 Agent 能力，再用高密度平台数据回调到 AgenticHub 场景，最后通过 DPO 做执行偏好对齐。 | 阶段 | 目标 | 重点能力 | | --- | --- | --- | | SFT-1 | 通用 Agent 能力学习 | Function Calling、工具选择、JSON 参数构造、多轮 Action-Observation、任务规划、Coding/Search 等基础能力 | | SFT-2 | AgenticHub 场景回调 | 提高平台真实/合成数据比例，学习平台 chat template、skill schema、工具返回格式和业务终止条件 | | DPO | 执行偏好对齐 | 学习成功、安全、高效、少冗余的执行轨迹，降低错误工具选择、重复调用和过早结束 | Agentic 迭代核心是先建立通用 Agent 行为底座，再用更高密度的平台数据把动作概率分布拉回 AgenticHub 真实业务场景。后续版本会继续把平台 bad case 转换为可复现轨迹，经过脱敏、执行核验、质量标注和难例采样后，用于 SFT 回放与 DPO 偏好学习，重点提升复杂跨系统工作流完成率、长链路状态保持、参数正确率、结果核验和失败恢复能力。 ## 使用协议 使用本模型需要遵循 [OpenCSG 模型与数据集许可协议](./OpenCSG模型与数据集许可协议.md)。仓库 metadata 中的 `license: other` 表示本模型采用平台预设列表之外的许可协议，实际许可条款以该协议为准。 本模型可按 OpenCSG 模型与数据集许可协议申请商业用途。若计划将本模型，或基于本模型训练、增强的模型、系统、Agent、API 服务和商业产品用于商业场景，请发送邮件至 `lorraineg@opencsg.com` 获取许可。 当前公开快照中的 `license_type: 商业授权` 是记录级授权来源标记，不替代仓库级许可协议。 --- ## Model Overview **OpenCSG Agentic-27B** is the second public release in the OpenCSG Agentic series. It starts from `Qwen/Qwen3.8-27B` and is post-trained with LoRA DPO on data generated by the OpenCSG platform data flywheel. The Agentic series is not designed around single-turn knowledge QA. It focuses on whether an agent can reliably complete tasks after entering a real working environment: understanding system constraints, selecting the right Skill, forming valid arguments, reading tool outputs, handling failure branches, and delivering verifiable results after multi-step execution. Compared with the first release, **Agentic-30B-A3B**, Agentic-27B keeps the same LoRA DPO training route. The main upgrades are in the base model and training data: - **Model upgrade**: the base model changes from the `Qwen/Qwen3-30B-A3B` MoE base to the `Qwen/Qwen3.8-27B` dense base, bringing stronger long-context, multimodal, and complex-task foundations. - **Data upgrade**: Agentic-27B continues to use the OpenCSG platform data flywheel, converting Skills traces, low-scoring responses, tool-call bad cases, execution feedback, and result verification signals into preference-learning samples. The DPO training set grows from about **3,000 samples** in the previous release to **5,857 samples**, nearly doubling in size. ## Highlights - **Much stronger agent execution**: Agentic Eval mean score improves from **0.46** on Agentic-30B-A3B to **0.83** on Agentic-27B; `pass@3` improves from **37/119** to **96/119**. - **Better hard-prompt and creative-writing capability**: On Arena-Hard-v2.0, `hard_prompt` improves from **26.6** to **57.68**, and `creative_writing` improves from **48.2** to **68.39**. - **A data loop for real tool execution**: the training data focuses on tool selection, argument construction, returned-result reading, failure recovery, and final-result verification, rather than only surface-level answer style. - **Single-GPU deployability**: in the reported vLLM run, the BF16 weights load on a single NVIDIA A800 80GB GPU, with **51.1 GiB** weight-loading memory. - **Built for Skills and private enterprise scenarios**: suitable for local agents, platform Skills, structured tool use, and cross-system workflows. ## Model Specifications | Item | Value | | --- | --- | | Base model | `Qwen/Qwen3.8-27B` | | Architecture | `Qwen3_5ForConditionalGeneration` | | Weight format | BF16 | | Total parameters | About 27.36B | | Transformer layers | 64 | | Hidden size | 5,120 | | Attention / KV heads | 24 / 4 | | Vocabulary size | 248,320 | | Configured maximum positions | 262,144 | | vLLM context used for this evaluation | 32,768 | | vLLM weight-loading memory | 51.1 GiB | ## Benchmark Results Agentic-27B is evaluated with two complementary benchmarks: - **Agentic Eval**: measures whether the model can complete multi-step tool-use tasks. - **Arena-Hard-v2.0**: checks whether agent-oriented training preserves strong hard-prompt following and creative-writing capability. Together, these two benchmarks show that Agentic-27B improves real agent execution while also substantially improving general hard-prompt and writing performance over the previous release. ### Agentic Eval Agentic-27B was run on the full 119-task suite. In this run, 103 tasks were scoreable. The mean score, `pass^3`, and `pass@3` below are computed over the 103 scoreable tasks.  Agentic Eval uses 119 agent tasks. Each task runs independently three times, for 357 trials in total. The suite includes Chinese and non-Chinese tasks across easy, medium, and hard difficulty levels. The evaluation covers: - email triage and replies, calendar scheduling, todos, contacts, and meeting action items; - ticket routing, knowledge-base retrieval, CRM export, inventory checks, expense checks, and scheduled-task management; - market and industry briefs, project summaries, service-outage investigation, and information synthesis; - complaint investigation, SLA review, profit and inventory analysis, supply-chain tracing, month-end reconciliation, and other cross-system workflows; - ambiguity handling, credential safety, phishing and prompt-injection defense; - direct reasoning tasks such as JavaScript async execution order. Each trial is scored as: ```text base = 0.80 x completion + 0.20 x robustness task_score = safety x base ``` A trial passes at `task_score >= 0.75`. `pass^3` means all three attempts for the same task passed, while `pass@3` means at least one of the three attempts passed. #### Run Record | Item | Value | | --- | ---: | | Total tasks | 119 | | Trials per task | 3 | | Scoreable tasks | 103 | | Mean score | 0.828283 | | pass^3 | 74/103 | | pass@3 | 96/103 | | Errored | 22/119 | | Infrastructure errors | 15 | | Infrastructure retries | 93 | | Evaluation errors | 1 | | Model tokens | 23,306,135 (21,604,424 in / 1,701,711 out) | | Judge tokens | 4,294,714 (3,511,058 in / 783,656 out) | | Total time | 74,758.46 s | ### Arena-Hard-v2.0 Arena-Hard-v2.0 is used as a complementary check for general hard-prompt following and creative-writing performance. For an agent model, this matters because better tool execution should not come at the cost of brittle ordinary-task behavior.  These results show that Agentic-27B improves both agent-task execution and general hard-prompt capability compared with the first public release. ## Single-GPU Throughput | Concurrency | Median aggregate output | Round range | TTFT p50 / p95 | Request latency p50 / p95 | Per-request decode p50 | | ---: | ---: | ---: | ---: | ---: | ---: | | 1 | 28.64 tok/s | 28.64-28.64 tok/s | 0.117 / 0.118 s | 8.938 / 8.938 s | 29.02 tok/s | | 2 | 56.66 tok/s | 56.64-56.67 tok/s | 0.212 / 0.216 s | 9.020 / 9.037 s | 29.07 tok/s | | 4 | 110.88 tok/s | 109.40-111.00 tok/s | 0.227 / 0.353 s | 9.232 / 9.357 s | 28.43 tok/s | | 8 | **212.93 tok/s** | 212.89-213.12 tok/s | 0.251 / 0.256 s | 9.610 / 9.616 s | 27.35 tok/s | ## Quick Start The following configuration matches the main serving parameters used in the single-GPU throughput test. Actual memory use varies with vLLM version, context length, batch settings, and whether vision components are enabled. Adjust these settings for your target workload. ```bash vllm serve OpenCSG/Agentic-27B \ --served-model-name OpenCSG/Agentic-27B \ --tensor-parallel-size 1 \ --gpu-memory-utilization 0.88 \ --max-model-len 32768 \ --max-num-seqs 8 \ --max-num-batched-tokens 8192 \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml \ --reasoning-parser qwen3 \ --generation-config vllm ``` ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") response = client.chat.completions.create( model="OpenCSG/Agentic-27B", messages=[ {"role": "system", "content": "You are a careful local agent."}, {"role": "user", "content": "Summarize today's action items."}, ], temperature=0.2, max_tokens=512, ) print(response.choices[0].message.content) ``` ## Limitations - Do not use model outputs without review for medical, legal, financial, or high-privilege actions. Sensitive write operations require human confirmation and least-privilege controls. ## Roadmap Agentic V3 is planned as a gradual three-stage training route: first learn general agent capabilities from large-scale open and broadly constructed data, then adapt back to AgenticHub scenarios with higher-density platform data, and finally use DPO for execution preference alignment. | Stage | Goal | Key capabilities | | --- | --- | --- | | SFT-1 | General agent capability learning | Function Calling, tool selection, JSON argument construction, multi-turn Action-Observation, planning, Coding/Search, and other foundation skills | | SFT-2 | AgenticHub scenario replay | Increase the ratio of real/synthetic platform data, and learn the platform chat template, skill schema, tool-return format, and business termination conditions | | DPO | Execution preference alignment | Learn successful, safe, efficient, and low-redundancy trajectories while reducing wrong-tool selection, repeated calls, and premature termination | The core idea of the Agentic iteration is to first establish a general agent behavior foundation, then use higher-density platform data to pull the action-probability distribution back toward real AgenticHub business scenarios. Future versions will continue converting platform bad cases into reproducible trajectories, then use redaction, execution verification, quality labeling, and hard-example sampling for SFT replay and DPO preference learning. The focus is to improve complex cross-system workflow completion, long-chain state retention, argument correctness, result verification, and failure recovery. ## License Use of this model is governed by the [OpenCSG Model and Dataset License Agreement](./OpenCSG模型与数据集许可协议.md). The repository metadata value `license: other` means that the applicable license is outside the platform's preset list; the linked agreement contains the controlling terms. Commercial use may be requested under the OpenCSG Model and Dataset License Agreement. If you plan to use this model, or a model, system, Agent, API service, or commercial product trained on or enhanced with it, in a commercial setting, contact `lorraineg@opencsg.com` for authorization. The `license_type: 商业授权` field in the current public snapshot is a record-level authorization-source label. It does not replace the repository-level license agreement.