---
title: webssl-mae300m-full2b-224
canonical_url: "https://www.modelscope.cn/models/facebook/webssl-mae300m-full2b-224"
md_url: "https://www.modelscope.cn/models/facebook/webssl-mae300m-full2b-224.md"
repository: facebook/webssl-mae300m-full2b-224
last_updated: 2025-05-20
license: cc-by-nc-4.0
model_type:
  - vit
architectures:
  - ViTModel
parameters: 304.4M
tensor_type:
  - F32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
downloads: 364
stars: 0
---

# webssl-mae300m-full2b-224

> webssl-mae300m-full2b-224 - facebook 在 ModelScope 开源的模型。Web-SSL MAE ViT-L (300M): 2B MetaCLIP data, 224 Resolution A 300 million parameter Vision Transformer (ViT-L) trained with Masked Autoencoder (MAE) self-supervised learning on web-scale image data…

facebook/webssl-mae300m-full2b-224 是 ModelScope 魔搭社区上的 304.4M 参数机器学习模型，采用 cc-by-nc-4.0 许可。

- **Repository**: facebook/webssl-mae300m-full2b-224
- **License**: cc-by-nc-4.0
- **Parameters**: 304.4M
- **Downloads**: 364
- **Stars**: 0
- **Last updated**: 2025-05-20

Source: https://www.modelscope.cn/models/facebook/webssl-mae300m-full2b-224

---

# Web-SSL MAE ViT-L (300M): 2B MetaCLIP data, 224 Resolution
A 300 million parameter Vision Transformer (ViT-L) trained with Masked Autoencoder (MAE) self-supervised learning on web-scale image data without language supervision. Introduced in ["Scaling Language-Free Visual Representation Learning"](https://arxiv.org/abs/2504.01017) (Fan et al., 2025).

## Model Details
- **Architecture**: ViT-L (Large)
- **Parameters**: 300M
- **Resolution**: 224×224 pixels
- **Training**: Self-supervised Web-MAE on 2B image samples from MetaCLIP web data

## Model Descriptions
Web-SSL MAE ViT-L is a 300 million parameter Vision Transformer model trained using masked autoencoder self-supervised learning on 2 billion web images without language supervision. This model demonstrates that pure visual learning, when scaled appropriately, can match or exceed the performance of language-supervised models like CLIP across various vision tasks. As part of the Web-SSL family, it provides an efficient option while still maintaining strong visual representation capabilities.

<img src="webssl_teaser.png" alt="WebSSL Model Overview" width="600">

## Usage
```python
from transformers import AutoImageProcessor, ViTModel
import torch
from PIL import Image

# Adjust the size, crop_size, etc. fields to your liking
processor = AutoImageProcessor.from_pretrained('facebook/webssl-mae300m-full2b-224')
model = ViTModel.from_pretrained('facebook/webssl-mae300m-full2b-224').cuda().eval()

# Process an image
image = Image.open('path/to/image.jpg')
inputs = processor(images=image, return_tensors="pt").to('cuda')
with torch.no_grad():
    outputs = model(**inputs)

# Extract features from the encoder
encoder_hidden_states = outputs.last_hidden_state
```

## Citation

```bibtex
@article{fan2025scaling,
  title={Scaling Language-Free Visual Representation Learning}, 
  author={David Fan and Shengbang Tong and Jiachen Zhu and Koustuv Sinha and Zhuang Liu and Xinlei Chen and Michael Rabbat and Nicolas Ballas and Yann LeCun and Amir Bar and Saining Xie},
  year={2025},
  eprint={2504.01017},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}
```