---
title: cv_panovit_indoor-layout-estimation
canonical_url: "https://www.modelscope.cn/models/iic/cv_panovit_indoor-layout-estimation"
md_url: "https://www.modelscope.cn/models/iic/cv_panovit_indoor-layout-estimation.md"
repository: iic/cv_panovit_indoor-layout-estimation
chinese_name: "全景图室内框架估计"
last_updated: 2023-02-16
license: "Apache License 2.0"
pipeline_tag: indoor-layout-estimation
tasks:
  - indoor-layout-estimation
model_type:
  - roberta
library_name:
  - pytorch
frameworks:
  - pytorch
domain:
  - cv
downloads: 9030
stars: 14
tags:
  - ViT
  - Alibaba
  - Layout
  - Panorama
  - XR
  - "arxiv:2212.12156"
---

# cv_panovit_indoor-layout-estimation

> cv_panovit_indoor-layout-estimation - iic 在 ModelScope 开源的模型。基于单张全景图估计房间的三维框架是实现基于图像的场景三维重建任务的关键技术之一。提出PanoViT，一种用于从单张全景图像估计房间布局的视觉自注意力模型。 与 CNN 模型相比，PanoViT更擅长学习全景图像中的全局关系，以更好估计复杂的房间框架结构。

iic/cv_panovit_indoor-layout-estimation 是 ModelScope 魔搭社区上的indoor-layout-estimation模型，采用 Apache License 2.0 许可。

- **Repository**: iic/cv_panovit_indoor-layout-estimation
- **License**: Apache License 2.0
- **Tasks**: indoor-layout-estimation
- **Tags**: ViT, Alibaba, Layout, Panorama, XR, arxiv:2212.12156
- **Downloads**: 9030
- **Stars**: 14
- **Last updated**: 2023-02-16

Source: https://www.modelscope.cn/models/iic/cv_panovit_indoor-layout-estimation

---

# 室内框架估计算法介绍

输入一张室内空间的全景RGB图像，室内框架估计算法将输出房间的墙线，天花板线跟地线
<p float="left">
  &emsp;&emsp; <img src="description/intro.png" width="1000" />
</p>

## 模型描述
PanoViT 的网络框架可以分为backbone、全景视觉变换器编码器（Vision transformer encoder）、边缘增强模块（Edge enhancement module）和布局预测模块（Layout prediction module）。一张全景图送到backbone提取多尺度特征图，送到边缘增强模块得到边缘增强图。全景视觉转换器编码器以原始图像、边缘增强图和多尺度特征图为输入，输出特征向量供布局预测模块估计房间布局。网络的结构如图所示

<p float="left">
  &emsp;&emsp; <img src="description/network.png" width="1000" />
</p>


## 如何使用

### 代码范例

```python
import cv2
from modelscope.outputs import OutputKeys
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks


task = Tasks.indoor_layout_estimation
model_id = 'damo/cv_panovit_indoor-layout-estimation'

input_location = 'data/test/images/indoor_layout_estimation.png'
estimator = pipeline(Tasks.indoor_layout_estimation, model=model_id)
result = estimator(input_location)
layout_vis = result[OutputKeys.LAYOUT]
cv2.imwrite('layout.jpg', layout_vis)
```

### 输入图像格式

输入图像与mattroport3D数据集一致，为512*1024的全景图像。


## 模型精度
在Mattroport上的结果为
<p float="left">
  &emsp;&emsp; <img src="description/result.png" width="1000" />
</p>


## Bibtex
```text
@article{shen2022panovit,
  title={PanoViT: Vision Transformer for Room Layout Estimation from a Single Panoramic Image},
  author={Shen, Weichao and Dong, Yuan and Chen, Zonghao and Zhao, Zhengyi and Gao, Yang and Liu, Zhu},
  journal={arXiv preprint arXiv:2212.12156},
  year={2022}
}
```