---
title: "FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates"
canonical_url: "https://www.modelscope.cn/papers/124596"
md_url: "https://www.modelscope.cn/papers/124596.md"
arxiv_id: 2503.07216
published: 2025-03-11
last_updated: 2025-03-11
authors:
  - "Sangwoo Park"
  - "Seanie Lee"
  - "Byungjoo Kim"
  - "Sung Ju Hwang"
model_name: FedRand
model_developer: "KAIST、DeepAuto.ai"
domain:
  - "机器学习"
  - "联邦学习"
  - "隐私保护"
  - "视觉语言模型"
  - "参数高效微调"
type:
  - "机器学习"
  - "联邦学习"
  - "隐私保护"
  - "视觉语言模型"
  - "参数高效微调"
  - "Machine Learning"
arxiv_url: "https://arxiv.org/abs/2503.07216"
pdf_url: "https://arxiv.org/pdf/2503.07216"
---

# FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates

> Federated Learning (FL) is a widely used framework for training models in a decentralized manner, ensuring that the central server does not have direct access to data from local clients. However, this approach may still fail to fully preserve data privacy,…

「FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates」是 ModelScope 魔搭社区收录的论文，arXiv 2503.07216，作者为 Sangwoo Park, Seanie Lee, Byungjoo Kim et al.，发表于 2025-03-11，属于 机器学习、联邦学习、隐私保护 领域。

- **ArXiv**: 2503.07216
- **Published**: 2025-03-11
- **Authors**: Sangwoo Park, Seanie Lee, Byungjoo Kim, Sung Ju Hwang
- **Model**: FedRand
- **Developer**: KAIST、DeepAuto.ai
- **Domain**: 机器学习, 联邦学习, 隐私保护, 视觉语言模型, 参数高效微调
- **ArXiv URL**: https://arxiv.org/abs/2503.07216
- **PDF**: https://arxiv.org/pdf/2503.07216

Source: https://www.modelscope.cn/papers/124596

---

> FedRand：通过随机化LoRA子参数更新增强联邦学习中的隐私保护

## 摘要

本文针对联邦学习（FL）在训练视觉-语言模型（VLMs）时存在的隐私问题，提出了一种增强隐私保护的框架——FedRand。传统的FL方法虽然避免了直接共享数据，但在参数聚合过程中仍可能暴露客户端模型参数，导致会员推断攻击（MIAs）。为解决这一问题，FedRand采用随机选择低秩适应（LoRA）子参数更新的方法，每个客户端仅发送部分非私有参数给服务器，从而防止完整模型参数的泄露。具体来说，FedRand通过随机选择A或B矩阵进行初始化，并使用前一轮的客户端参数作为私有参数。实验结果表明，FedRand不仅提高了对MIAs的鲁棒性，还保持了与完全传输LoRA参数方法相当的准确性，同时减少了通信成本。该方法在多个基准数据集上进行了验证，包括ScienceQA、MSCOCO和NoCaps，展示了其在视觉问答和图像字幕任务中的有效性。

## Abstract

Federated Learning (FL) is a widely used framework for training models in a decentralized manner, ensuring that the central server does not have direct access to data from local clients. However, this approach may still fail to fully preserve data privacy, as models from local clients are exposed to the central server during the aggregation process. This issue becomes even more critical when training vision-language models (VLMs) with FL, as VLMs can easily memorize training data instances, making them vulnerable to membership inference attacks (MIAs). To address this challenge, we propose the FedRand framework, which avoids disclosing the full set of client parameters. In this framework, each client randomly selects subparameters of Low-Rank Adaptation (LoRA) from the server and keeps the remaining counterparts of the LoRA weights as private parameters. After training both parameters on the client's private dataset, only the non-private client parameters are sent back to the server for aggregation. This approach mitigates the risk of exposing client-side VLM parameters, thereby enhancing data privacy. We empirically validate that FedRand improves robustness against MIAs compared to relevant baselines while achieving accuracy comparable to methods that communicate full LoRA parameters across several benchmark datasets.