---
title: webssl-dino300m-light2b-224
canonical_url: "https://www.modelscope.cn/models/facebook/webssl-dino300m-light2b-224"
md_url: "https://www.modelscope.cn/models/facebook/webssl-dino300m-light2b-224.md"
repository: facebook/webssl-dino300m-light2b-224
last_updated: 2026-03-21
license: cc-by-nc-4.0
model_type:
  - dinov2
architectures:
  - Dinov2Model
parameters: 303.7M
tensor_type:
  - F32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
downloads: 43
stars: 0
---

# webssl-dino300m-light2b-224

> webssl-dino300m-light2b-224 - facebook 在 ModelScope 开源的模型。Web-SSL DINO ViT-300M: 2B MetaCLIP data, 224 Resolution A 300 million parameter Vision Transformer (ViT) trained with DINOv2 self-supervised learning on lightly filtered web-scale image data without…

facebook/webssl-dino300m-light2b-224 是 ModelScope 魔搭社区上的 303.7M 参数机器学习模型，采用 cc-by-nc-4.0 许可。

- **Repository**: facebook/webssl-dino300m-light2b-224
- **License**: cc-by-nc-4.0
- **Parameters**: 303.7M
- **Downloads**: 43
- **Stars**: 0
- **Last updated**: 2026-03-21

Source: https://www.modelscope.cn/models/facebook/webssl-dino300m-light2b-224

---

# Web-SSL DINO ViT-300M: 2B MetaCLIP data, 224 Resolution
A 300 million parameter Vision Transformer (ViT) trained with DINOv2 self-supervised learning on lightly filtered web-scale image data without language supervision. Introduced in ["Scaling Language-Free Visual Representation Learning"](https://arxiv.org/abs/2504.01017) (Fan et al., 2025).

## Model Details
- **Architecture**: ViT (1536 width, 40 depth, 24 heads)
- **Parameters**: 300M
- **Resolution**: 224×224 pixels
- **Training**: Self-supervised Web-DINO on lightly filtered MetaCLIP data

## Model Descriptions
Web-SSL DINO 2B is a 300 million parameter Vision Transformer model trained using self-supervised learning on lightly filtered web images without language supervision. The "light2b" designation indicates training on a subset of images containing any textual content, retaining approximately 50.3% of the original MetaCLIP dataset. This filtering improves OCR & Chart understanding capabilities while maintaining strong performance across all vision tasks. This model demonstrates that pure visual learning, when scaled appropriately, can match or exceed the performance of language-supervised models like CLIP across various vision tasks.
<img src="webssl_teaser.png" alt="WebSSL Model Overview" width="600">

## Usage
```python
from transformers import AutoImageProcessor, Dinov2Model
import torch
from PIL import Image

processor = AutoImageProcessor.from_pretrained('facebook/webssl-dino300m-light2b-224')
model = Dinov2Model.from_pretrained('facebook/webssl-dino300m-light2b-224')

# Process an image
image = Image.open('path/to/image.jpg')
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

cls_features = outputs.last_hidden_state[:, 0]  # CLS token features
patch_features = outputs.last_hidden_state[:, 1:] # patch-wise token features
```

## Citation

```bibtex
@article{fan2025scaling,
  title={Scaling Language-Free Visual Representation Learning}, 
  author={David Fan and Shengbang Tong and Jiachen Zhu and Koustuv Sinha and Zhuang Liu and Xinlei Chen and Michael Rabbat and Nicolas Ballas and Yann LeCun and Amir Bar and Saining Xie},
  year={2025},
  eprint={2504.01017},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}
```