---
title: cv_human_68-facial-landmark-detection
canonical_url: "https://www.modelscope.cn/models/Damo_XR_Lab/cv_human_68-facial-landmark-detection"
md_url: "https://www.modelscope.cn/models/Damo_XR_Lab/cv_human_68-facial-landmark-detection.md"
repository: Damo_XR_Lab/cv_human_68-facial-landmark-detection
chinese_name: "真人人脸68个关键点检测"
last_updated: 2024-11-11
license: "Apache License 2.0"
pipeline_tag: face-2d-keypoints
tasks:
  - face-2d-keypoints
library_name:
  - pytorch
frameworks:
  - Pytorch
downloads: 3423
stars: 14
---

# cv_human_68-facial-landmark-detection

> cv_human_68-facial-landmark-detection - Damo_XR_Lab 在 ModelScope 开源的模型。该模型主要用于人脸2D关键点检测任务，返回68个人脸关键点2D坐标

Damo_XR_Lab/cv_human_68-facial-landmark-detection 是 ModelScope 魔搭社区上的face-2d-keypoints模型，采用 Apache License 2.0 许可。

- **Repository**: Damo_XR_Lab/cv_human_68-facial-landmark-detection
- **License**: Apache License 2.0
- **Tasks**: face-2d-keypoints
- **Downloads**: 3423
- **Stars**: 14
- **Last updated**: 2024-11-11

Source: https://www.modelscope.cn/models/Damo_XR_Lab/cv_human_68-facial-landmark-detection

---

# 任务
输入一张包含人脸图像（通过人脸检测模型crop出来或者手动crop），进行人脸2D关键点检测，输出人脸68个关键点的2D坐标。

# 68点人脸关键点定义
![](./assets/68_definition.png)

# 模型描述
该模型主要主要基于STAR loss(CVPR2023)网络，通过扩充训练数据集，优化数据增强部分，调整训练策略方式, 在300W等人脸关键点公开数据集上指标上达到SOTA。

# 训练数据
该模型主要使用学界300W，300VW, FaceSynthetics等包含68个人脸关键点标注信息的手动标注数据及渲染数据作为训练数据。

# 指标及可视化效果对比
## 指标对比
主要在300W数据集 (https://ibug.doc.ic.ac.uk/resources/300-W/) 上进行对比。
![](./assets/compare_table.png)

## 可视化效果对比
与目前学界较好的算法进行可视化效果对比，如下图所示，提供的模型特别在大角度(如半侧脸，侧脸)2D关键点检测更为精确。
![](./assets/compare_visualize.png)

# 模型用途
可用于真人人脸关键点检测场景，该模型已用于阿里巴巴通义实验室产品MotionShop[链接](https://modelscope.cn/studios/Damo_XR_Lab/motionshop)

# 安装及使用方式
#### 您可以通过如下git clone命令，或者ModelScope SDK来下载模型
SDK下载
```bash
# 安装ModelScope, 建议git安装
git clone https://github.com/modelscope/modelscope.git
cd modelscope
pip install -e .
```
```python
# SDK模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Damo_XR_Lab/cv_human_68-facial-landmark-detection')

```

## 代码示例
#### 已经将人脸区域crop出来,同时将图片resize为256*256大小
```python
import cv2
import copy
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

model_id = 'Damo_XR_Lab/cv_human_68-facial-landmark-detection'
estimator = pipeline(Tasks.facial_68ldk_detection, model=model_id)

Input_file = 'assets/sample.jpg'
cv_img = cv2.imread(Input_file)
cv_img = cv2.resize(cv_img, (256, 256))

results = estimator(input=cv_img)
landmarks = results['landmarks']

image_draw = copy.copy(cv_img)
for num in range(landmarks.shape[0]):
    cv2.circle(image_draw, (round(landmarks[num][0]), round(landmarks[num][1])), 2, (0, 255, 0), -1)
cv2.imwrite('result.png', image_draw)
```

* 输入说明： 输入数据Input_file是人脸图像文件位置或者opencv-python读取的numpy格式数据
* 输出说明： 输出数据landmarks是numpy格式的二维数组


## 引用
``` Bibtex
@inproceedings{Zhou_2023_CVPR,
    author    = {Zhou, Zhenglin and Li, Huaxia and Liu, Hong and Wang, Nanyang and Yu, Gang and Ji, Rongrong},
    title     = {STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {15475-15484}
}
@inproceedings{Prados-Torreblanca_2022_BMVC,
  author    = {Andrés  Prados-Torreblanca and José M Buenaposada and Luis Baumela},
  title     = {Shape Preserving Facial Landmarks with Graph Attention Networks},
  booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
  publisher = {{BMVA} Press},
  year      = {2022},
  url       = {https://bmvc2022.mpi-inf.mpg.de/0155.pdf}
}
@article{JLS21,
  title={Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild},
  author={Haibo Jin and Shengcai Liao and Ling Shao},
  journal={International Journal of Computer Vision},
  publisher={Springer Science and Business Media LLC},
  ISSN={1573-1405},
  url={http://dx.doi.org/10.1007/s11263-021-01521-4},
  DOI={10.1007/s11263-021-01521-4},
  year={2021},
  month={Sep}
}

```