--- title: laya-vulnerabilit canonical_url: "https://www.modelscope.cn/models/xiahao90/laya-vulnerabilit" md_url: "https://www.modelscope.cn/models/xiahao90/laya-vulnerabilit.md" repository: xiahao90/laya-vulnerabilit last_updated: 2026-10-12 license: "Apache License 2.0" base_model: - convaiinnovations/laya base_model_relation: finetune parameters: 421.3M tensor_type: - F32 library_name: - safetensors language: - en - zh downloads: 8 stars: 1 tags: - laya - vulnerability-detection - experimental --- # laya-vulnerabilit > laya-vulnerabilit - xiahao90 在 ModelScope 开源的模型。Laya vulnerability detection — experimental checkpoint xiahao90/laya-vulnerabilit 是 ModelScope 魔搭社区上的 421.3M 参数机器学习模型，采用 Apache License 2.0 许可，基于 convaiinnovations/laya 构建。 - **Repository**: xiahao90/laya-vulnerabilit - **License**: Apache License 2.0 - **Parameters**: 421.3M - **Base model**: convaiinnovations/laya - **Tags**: laya, vulnerability-detection, experimental - **Downloads**: 8 - **Stars**: 1 - **Last updated**: 2026-10-12 Source: https://www.modelscope.cn/models/xiahao90/laya-vulnerabilit --- # Laya vulnerability detection — experimental checkpoint This checkpoint is a supervised adaptation of [Convai Innovations Laya](https://huggingface.co/convaiinnovations/laya), using its original decision architecture. The base model is Apache-2.0. This derivative checkpoint and the publisher-authored code and documentation are released under Apache-2.0. Upstream components retain their original licenses and attribution; see LICENSE and NOTICE. This package is prepared for ModelScope repository `xiahao90/laya-vulnerabilit`; upload status is recorded separately. ## Training scope - Method: cross-entropy, `tail` mode; not RLCD reproduction. - Actual training samples: 7506; best epoch: 3. - Completed epochs: 3; optimizer updates: 5631; trainable parameters: 50763777 of 421293827. - Selected training languages: bash, c, cpp, go, java, javascript, php, python, ruby, rust, typescript. - Context: 512 tokens including the question; longer samples were excluded, not mislabeled after truncation. - Actual device: `mps`; seed: 42; learning rate: 3e-05. - Frozen act/escalate policy: its output is not a trained vulnerability confidence. - Base SHA-256: `891102d372688fc2a094dac56a384bc537b87c63f21f9f3dac0be2b7cbc8d86c`. - Output SHA-256: `c351ffebcf432a40cfa929613c2063e4f98d49c3de8868a9a901cc708dc3e5c7`. ## Training data - benchproctor: https://github.com/TheAuditorTool/BenchProctor, revision `ad70ff85c2325626ccbda2a6565d354732d032ff`, repository license `Apache-2.0`. Source-repository licenses do not automatically relicense third-party code. Original artifacts, notices, exact source paths and label scopes are retained locally. Metadata/CWE/CVE labels are not model inputs. Exact duplicates and conflicting labels are handled before connected-family splitting. ## Evaluation Test samples: 495. These are sampled, context-filtered holdouts; this is not the full upstream benchmark. Validation selects the epoch, temperature and decision threshold. Test predictions are recorded only for evaluation. Baseline metrics use temperature 1.0; final probabilities use validation-fitted calibration. | Metric at probability threshold 0.5 | Base | Fine-tuned | |---|---:|---:| | Accuracy | 0.4727 | 0.7414 | | Precision | 0.4082 | 0.7296 | | Recall | 0.0794 | 0.7817 | | False-positive rate | 0.1193 | 0.3004 | | F1 | 0.1329 | 0.7548 | | PR-AUC / average precision | 0.4883 | 0.7978 | | ROC-AUC | 0.4715 | 0.8175 | | Brier | 0.3471 | 0.1707 | Validation-fitted temperature: 1.694221. Threshold targeting validation FPR <= 5.0%: 0.759043. Test recall at this threshold: 0.2738; test FPR: 0.0370. A threshold above 1 means all predictions are negative, not a successful detector. A validation FPR target is not a guarantee on unseen projects. ### Independent source: devign Actual evaluated samples: 611; selected before context filtering: 1024. Known sample IDs and structural fingerprints from this run were excluded. No external-source tuning was performed. | Metric at threshold 0.5 | Base | Fine-tuned | |---|---:|---:| | Accuracy | 0.5041 | 0.4943 | | F1 | 0.1465 | 0.5554 | | PR-AUC / average precision | 0.4756 | 0.5012 | | ROC-AUC | 0.4625 | 0.5003 | At the frozen validation threshold: recall 0.0700, FPR 0.0643. Balanced, context-filtered function samples; upstream negatives are not proof of complete safety. Not a project-level audit. Observed ROC-AUC is near chance; this experiment does not establish useful discrimination on this external source. A higher F1 caused by predicting more positives is not evidence of reliable detection. ## Limitations Experimental, not a production scanner. Small or synthetic data can produce near-random detection even when calibration improves. No reliable vulnerability explanations, line localization, cross-file reasoning, long-file coverage, or real-project accuracy is established. Safe labels mean the benchmark's target weakness is absent or the upstream function is not known vulnerable, not that all possible vulnerabilities are absent. Source normalization uses heuristic lexers; historical experiment snapshots normalized blank lines and trailing whitespace. Exact model inputs are retained, but full semantic equivalence to every original language construct is not established. ## Usage Load with `laya.load(local_checkpoint_path)`. Use the exact `training.task.QUESTION` with `training.task.state_for(record)`; changing question wording is an untested distribution shift. The existing HTTP API can load this directory using `MODEL_DIR`. Full configuration, data hashes, chosen samples and epoch logs are in `training-records/` and `training-data/`. The SDK's `choice` uses argmax (threshold 0.5), not the custom validation threshold. To apply the recorded operating point, compare `probabilities["vulnerable"]` with `agent.cfg["vulnerability_threshold"]`. A threshold above 1 is unusable for detection. ## 中文说明与 ModelScope 使用 本模型是 Laya 的代码漏洞二分类实验微调版，保留原始决策架构；不是生成式聊天模型。任务输出为 `safe` 和 `vulnerable` 的概率。当前实验没有达到 90% 准确率：BenchProctor 合成测试 495 条的准确率为 74.14%，Devign 外部真实函数 611 条为 49.43%，ROC-AUC 为 0.5003。真实项目上的有效检测能力尚未得到证实。 ### 下载与安装 仓库 ID 按发布者提供的名称使用 `xiahao90/laya-vulnerabilit`（末尾没有 y）。在独立虚拟环境中运行： ```bash python -m venv .venv source .venv/bin/activate python -m pip install modelscope-hub laya==0.3.5 ms-hub download xiahao90/laya-vulnerabilit --repo-type model --local-dir ./laya-vulnerability ``` 私有仓库需要先用 `ms-hub login` 登录有读取权限的账号。完全复现训练所用依赖见 `training/requirements-training.txt`，不同平台的 PyTorch 安装方式和可用设备可能不同。 ### 本地推理 在下载的模型目录内运行下例；输入仅包含语言和代码，不应包含答案、CWE/CVE 标签或修复说明。支持语言名称为 bash、c、cpp、go、java、javascript、php、python、ruby、rust、typescript。 ```python import laya from pathlib import Path from training.task import QUESTION, state_for model_dir = Path(".").resolve() # 在下载的模型目录内运行 agent = laya.load(str(model_dir), device="cpu") record = {"language": "c", "code": "int identity(int x) { return x; }"} result = agent.predict(state_for(record), {"vulnerability": QUESTION}) answer = result["answers"]["vulnerability"] p = float(answer["probabilities"]["vulnerable"]) threshold = float(agent.cfg["vulnerability_threshold"]) print({"probability_vulnerable": p, "threshold": threshold, "prediction": "vulnerable" if p >= threshold else "safe"}) ``` 上例需在下载模型目录内运行，使随包 `training.task` 可导入。`QUESTION` 必须保持训练时的原文。输入应采用训练数据对应的代码规范化方式；直接输入任意原始文件的效果未经此实验验证。SDK 的 `choice` 使用最大概率选项，相当于二分类阈值 0.5；上例显式使用验证集选出的 0.7590428。该阈值在合成测试上的召回率仅 27.38%，误报率 3.70%；在 Devign 上召回率 7.00%，误报率 6.43%。提高阈值不能替代识别能力，也不能把概率解释为经过真实项目验证的安全保证。`act/escalate` 策略未在本次训练中更新。 ### 输入长度与使用范围 训练总序列长度为 512 tokens，包含问题和代码；长样本在训练和评估前被排除。调用 SDK 不应将长文件自动截断后的答案当作完整文件结论。本模型未验证跨文件分析、漏洞定位、修复生成或全项目审计能力。`safe` 只表示当前任务没有展示对应漏洞，并不证明代码不存在其他漏洞。 ### 数据和训练范围 实际训练集 7,506 条，验证集 467 条，合成测试集 495 条，均来自固定版本 BenchProctor。仓库中虽收集了其他真实语料，但这些语料没有参与此权重的训练；Devign 只用于外部评估。训练解冻最后两层编码器及决策模块，训练参数 50,763,777，总参数 421,293,827，3 个 epoch、5,631 次更新，学习率 3e-5，batch size 4，seed 42，设备 Apple MPS。最佳 epoch、温度及阈值通过验证集确定，测试集没有用于选择参数。 ### 文件与可复现性 - `model.safetensors`：完整微调权重，约 1.685 GB，不是仅包含 LoRA 的适配器。 - `rl_agent_config.json`、`encoder/`、`tokenizer/`：Laya 加载所需配置和分词器。 - `training-data/`：本实验实际使用的合成样本及顺序，不是完整百万条语料。 - `training-records/`：训练日志、指标、来源追踪和权重变化审计。 - `training/`、`REPRODUCE.md`、`reproduction-recipe.json`：训练源码、依赖和复现命令。 - `source-notices/`、`BASE_MODEL_README.md`：数据来源声明和基础模型署名。 - `release-inventory.json`：随包文件大小及 SHA-256，用于下载后检查完整性；清单不包含自身。 ### 许可证与平台兼容 基础模型标注 Apache-2.0，BenchProctor 仓库标注 Apache-2.0，相关声明随包保留。本次衍生模型和发布者编写的训练代码、文档采用 Apache-2.0，允许商用、修改和再分发，不附加非商业或用途限制。上游组成部分及训练样本仍保留其原始许可和署名，详情见 LICENSE、NOTICE、BASE_MODEL_README.md 和 source-notices/。Apache-2.0 的声明不取消第三方已有权利或许可条件。本包不包含 Devign 原始代码，也不包含优化器状态或访问令牌。 ModelScope 用于托管和下载此自定义 Laya 模型。不能直接假设 `AutoModelForCausalLM` 或平台标准 `pipeline` 能加载；在线推理需要单独适配 Laya SDK。本发布保持实验定位，后续版本应单独记录训练数据、权重哈希和未参与调参的外部测试。 ### 命令行示例与长度检查 在下载目录内运行 `python infer.py --code-file /path/to/function.c --language c --device cpu`。此脚本在超过已训练的总序列长度时明确拒绝推理，避免静默截断。它仅提供基本加载和输入长度检查，不自动完成全语言代码规范化；评估一致性需要按训练预处理流程准备输入。