--- title: AIE-51-8-Law-Model canonical_url: "https://www.modelscope.cn/models/lingminai/AIE-51-8-Law-Model" md_url: "https://www.modelscope.cn/models/lingminai/AIE-51-8-Law-Model.md" repository: lingminai/AIE-51-8-Law-Model chinese_name: "AIE-51-8法律大语言模型" last_updated: 2025-01-01 license: other pipeline_tag: question-answering tasks: - question-answering model_type: - qwen2 architectures: - Qwen2ForCausalLM parameters: 3.1B tensor_type: - BF16 library_name: - safetensors - pytorch frameworks: - Pytorch inference_backends: - "deploy_task text/emb" - "lmdeploy 0.9.1" - "lmdeploy_turbomind 0.9.1" - "sglang 0.5.2" - "vllm 0.9.2" downloads: 100 stars: 6 tags: - llama-factory - full - generated_from_trainer --- # AIE-51-8-Law-Model > AIE-51-8-Law-Model - lingminai 在 ModelScope 开源的模型。AIE-51-8-Law-Model，AIE-51-8法律大语言模型 基于Qwen2.5-3B-Instruct模型微调训练的模型 lingminai/AIE-51-8-Law-Model 是 ModelScope 魔搭社区上的 3.1B 参数question-answering模型，采用 other 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。 - **Repository**: lingminai/AIE-51-8-Law-Model - **License**: other - **Tasks**: question-answering - **Parameters**: 3.1B - **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2 - **Tags**: llama-factory, full, generated_from_trainer - **Downloads**: 100 - **Stars**: 6 - **Last updated**: 2025-01-01 Source: https://www.modelscope.cn/models/lingminai/AIE-51-8-Law-Model --- # Qwen2.5-3B-Instruct-sft-allpair This model is a fine-tuned version of [/gemini/pretrain/Qwen2.5-3B-Instruct](https://huggingface.co//gemini/pretrain/Qwen2.5-3B-Instruct) on the law_sft_pair, the identity and the alpaca_zh_demo datasets. It achieves the following results on the evaluation set: - Loss: 0.4862 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - distributed_type: multi-GPU - num_devices: 2 - total_train_batch_size: 8 - total_eval_batch_size: 8 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 2.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:-----:|:---------------:| | 0.6862 | 0.0265 | 500 | 0.6764 | | 0.7135 | 0.0530 | 1000 | 0.6080 | | 0.6275 | 0.0794 | 1500 | 0.5907 | | 0.6341 | 0.1059 | 2000 | 0.5841 | | 0.6358 | 0.1324 | 2500 | 0.5837 | | 0.474 | 0.1589 | 3000 | 0.5825 | | 0.7276 | 0.1854 | 3500 | 0.5821 | | 0.6027 | 0.2118 | 4000 | 0.5838 | | 0.6226 | 0.2383 | 4500 | 0.5774 | | 0.4801 | 0.2648 | 5000 | 0.5794 | | 0.6302 | 0.2913 | 5500 | 0.5656 | | 0.5484 | 0.3177 | 6000 | 0.5701 | | 0.4836 | 0.3442 | 6500 | 0.5601 | | 0.5169 | 0.3707 | 7000 | 0.5563 | | 0.5769 | 0.3972 | 7500 | 0.5535 | | 0.5467 | 0.4237 | 8000 | 0.5533 | | 0.4572 | 0.4501 | 8500 | 0.5467 | | 0.5652 | 0.4766 | 9000 | 0.5453 | | 0.5942 | 0.5031 | 9500 | 0.5424 | | 0.544 | 0.5296 | 10000 | 0.5376 | | 0.7179 | 0.5561 | 10500 | 0.5377 | | 0.5242 | 0.5825 | 11000 | 0.5334 | | 0.6293 | 0.6090 | 11500 | 0.5333 | | 0.5513 | 0.6355 | 12000 | 0.5320 | | 0.5026 | 0.6620 | 12500 | 0.5308 | | 0.5034 | 0.6884 | 13000 | 0.5257 | | 0.5532 | 0.7149 | 13500 | 0.5233 | | 0.4264 | 0.7414 | 14000 | 0.5187 | | 0.5129 | 0.7679 | 14500 | 0.5158 | | 0.5232 | 0.7944 | 15000 | 0.5163 | | 0.5371 | 0.8208 | 15500 | 0.5120 | | 0.5421 | 0.8473 | 16000 | 0.5089 | | 0.4809 | 0.8738 | 16500 | 0.5088 | | 0.5588 | 0.9003 | 17000 | 0.5068 | | 0.4346 | 0.9268 | 17500 | 0.5059 | | 0.5689 | 0.9532 | 18000 | 0.5040 | | 0.6198 | 0.9797 | 18500 | 0.5017 | | 0.329 | 1.0062 | 19000 | 0.5113 | | 0.3231 | 1.0327 | 19500 | 0.5137 | | 0.3123 | 1.0592 | 20000 | 0.5156 | | 0.3361 | 1.0856 | 20500 | 0.5151 | | 0.2736 | 1.1121 | 21000 | 0.5129 | | 0.3009 | 1.1386 | 21500 | 0.5129 | | 0.2946 | 1.1651 | 22000 | 0.5100 | | 0.2651 | 1.1915 | 22500 | 0.5076 | | 0.4446 | 1.2180 | 23000 | 0.5070 | | 0.3746 | 1.2445 | 23500 | 0.5056 | | 0.3153 | 1.2710 | 24000 | 0.5076 | | 0.3054 | 1.2975 | 24500 | 0.5022 | | 0.3645 | 1.3239 | 25000 | 0.5000 | | 0.3244 | 1.3504 | 25500 | 0.4996 | | 0.2931 | 1.3769 | 26000 | 0.4995 | | 0.3572 | 1.4034 | 26500 | 0.4994 | | 0.2883 | 1.4299 | 27000 | 0.4992 | | 0.3109 | 1.4563 | 27500 | 0.4951 | | 0.3988 | 1.4828 | 28000 | 0.4940 | | 0.293 | 1.5093 | 28500 | 0.4965 | | 0.3099 | 1.5358 | 29000 | 0.4940 | | 0.2674 | 1.5623 | 29500 | 0.4940 | | 0.267 | 1.5887 | 30000 | 0.4920 | | 0.2835 | 1.6152 | 30500 | 0.4920 | | 0.4249 | 1.6417 | 31000 | 0.4904 | | 0.2893 | 1.6682 | 31500 | 0.4905 | | 0.363 | 1.6946 | 32000 | 0.4891 | | 0.37 | 1.7211 | 32500 | 0.4878 | | 0.4262 | 1.7476 | 33000 | 0.4882 | | 0.3343 | 1.7741 | 33500 | 0.4862 | | 0.3784 | 1.8006 | 34000 | 0.4871 | | 0.2619 | 1.8270 | 34500 | 0.4874 | | 0.3448 | 1.8535 | 35000 | 0.4865 | | 0.2832 | 1.8800 | 35500 | 0.4863 | | 0.3034 | 1.9065 | 36000 | 0.4863 | | 0.4282 | 1.9330 | 36500 | 0.4863 | | 0.3185 | 1.9594 | 37000 | 0.4863 | | 0.3791 | 1.9859 | 37500 | 0.4862 | ### Framework versions - Transformers 4.46.1 - Pytorch 2.1.2+cu121 - Datasets 2.18.0 - Tokenizers 0.20.3