---
title: "UniF$^2$ace: A Unified Fine-grained Face Understanding and Generation Model"
canonical_url: "https://www.modelscope.cn/papers/125793"
md_url: "https://www.modelscope.cn/papers/125793.md"
arxiv_id: 2503.08120
published: 2026-01-13
last_updated: 2026-01-13
authors:
  - "Junzhe Li"
  - "Sifan Zhou"
  - "Liya Guo"
  - "Xuerui Qiu"
  - "Linrui Xu"
  - "Delin Qu"
  - "Tingting Long"
  - "Chun Fan"
  - "Ming Li"
  - "Hehe Fan"
  - "Jun Liu"
  - "Shuicheng Yan"
model_name: UniF2ace
model_developer: "Peking University、Carnegie Mellon University、Tsinghua University、Institute of Automation, Chinese Academy of Sciences、Central South University、Fudan University、Guangming Lab、Zhejiang University、Lancaster University、National University of Singapore"
domain:
  - "计算机视觉"
  - "多模态学习"
  - "人脸理解"
  - "人脸生成"
  - "混合专家模型"
type:
  - "计算机视觉"
  - "多模态学习"
  - "人脸理解"
  - "人脸生成"
  - "混合专家模型"
  - "Computer Vision and Pattern Recognition"
  - "Artificial Intelligence"
  - "Machine Learning"
  - Multimedia
arxiv_url: "https://arxiv.org/abs/2503.08120"
pdf_url: "https://arxiv.org/pdf/2503.08120"
code_link: "https://github.com/tulvgengenr/UniF2ace"
---

# UniF$^2$ace: A Unified Fine-grained Face Understanding and Generation Model

> Unified multimodal models (UMMs) have emerged as a powerful paradigm in fundamental cross-modality research, demonstrating significant potential in both image understanding and generation. However, existing research in the face domain primarily faces two…

「UniF$^2$ace: A Unified Fine-grained Face Understanding and Generation Model」是 ModelScope 魔搭社区收录的论文，arXiv 2503.08120，作者为 Junzhe Li, Sifan Zhou, Liya Guo et al.，发表于 2026-01-13，属于 计算机视觉、多模态学习、人脸理解 领域。

- **ArXiv**: 2503.08120
- **Published**: 2026-01-13
- **Authors**: Junzhe Li, Sifan Zhou, Liya Guo, Xuerui Qiu, Linrui Xu, Delin Qu, Tingting Long, Chun Fan, Ming Li, Hehe Fan, Jun Liu, Shuicheng Yan
- **Model**: UniF2ace
- **Developer**: Peking University、Carnegie Mellon University、Tsinghua University、Institute of Automation, Chinese Academy of Sciences、Central South University、Fudan University、Guangming Lab、Zhejiang University、Lancaster University、National University of Singapore
- **Domain**: 计算机视觉, 多模态学习, 人脸理解, 人脸生成, 混合专家模型
- **ArXiv URL**: https://arxiv.org/abs/2503.08120
- **PDF**: https://arxiv.org/pdf/2503.08120
- **Code**: https://github.com/tulvgengenr/UniF2ace

Source: https://www.modelscope.cn/papers/125793

---

> UniF2ace：统一的细粒度人脸理解与生成模型

## 摘要

UniF2ace 是首个专为细粒度人脸理解与生成设计的统一多模态模型（UMM）。该模型提出双离散扩散（D3Diff）损失函数，将掩码生成模型与离散分数匹配扩散相统一，从而更精确地逼近负对数似然；同时引入多层分组混合专家（MoE）架构，在词元级和序列级自适应融合语义与身份特征，缓解属性遗忘现象。此外，作者构建了包含约13万图文对和100万视觉问答对的大规模数据集 UniF2aceD-1M。实验表明，UniF2ace 在多项人脸理解与生成基准上显著优于同等规模的现有模型。

## Abstract

Unified multimodal models (UMMs) have emerged as a powerful paradigm in fundamental cross-modality research, demonstrating significant potential in both image understanding and generation. However, existing research in the face domain primarily faces two challenges: $\textbf{(1)}$ $\textbf{fragmentation development}$, with existing methods failing to unify understanding and generation into a single one, hindering the way to artificial general intelligence. $\textbf{(2) lack of fine-grained facial attributes}$, which are crucial for high-fidelity applications. To handle those issues, we propose $\textbf{UniF$^2$ace}$, $\textit{the first UMM specifically tailored for fine-grained face understanding and generation}$. $\textbf{First}$, we introduce a novel theoretical framework with a Dual Discrete Diffusion (D3Diff) loss, unifying masked generative models with discrete score matching diffusion and leading to a more precise approximation of the negative log-likelihood. Moreover, this D3Diff significantly enhances the model's ability to synthesize high-fidelity facial details aligned with text input. $\textbf{Second}$, we propose a multi-level grouped Mixture-of-Experts architecture, adaptively incorporating the semantic and identity facial embeddings to complement the attribute forgotten phenomenon in representation evolvement. $\textbf{Finally}$, to this end, we construct UniF$^2$aceD-1M, a large-scale dataset comprising 130K fine-grained image-caption pairs and 1M visual question-answering pairs, spanning a much wider range of facial attributes than existing datasets. Extensive experiments demonstrate that UniF$^2$ace outperforms existing models with a similar scale in both understanding and generation tasks, with 7.1\% higher Desc-GPT and 6.6\% higher VQA-score, respectively.
