---
title: tipsv2-b14
canonical_url: "https://www.modelscope.cn/models/google/tipsv2-b14"
md_url: "https://www.modelscope.cn/models/google/tipsv2-b14.md"
repository: google/tipsv2-b14
last_updated: 2026-04-11
license: apache-2.0
model_type:
  - tipsv2
architectures:
  - TIPSv2Model
parameters: 195.9M
tensor_type:
  - F32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
downloads: 62
stars: 8
tags:
  - vision
  - image-text
  - contrastive-learning
  - zero-shot
  - feature-extraction
---

# tipsv2-b14

> tipsv2-b14 - google 在 ModelScope 开源的模型。TIPSv2 (Text-Image Pre-training with Spatial awareness) is a family of contrastive vision-language models that produce spatially rich image features aligned with text embeddings. This is the Base variant with 86M vision…

google/tipsv2-b14 是 ModelScope 魔搭社区上的 195.9M 参数机器学习模型，采用 apache-2.0 许可。

- **Repository**: google/tipsv2-b14
- **License**: apache-2.0
- **Parameters**: 195.9M
- **Tags**: vision, image-text, contrastive-learning, zero-shot, feature-extraction
- **Downloads**: 62
- **Stars**: 8
- **Last updated**: 2026-04-11

Source: https://www.modelscope.cn/models/google/tipsv2-b14

---

# TIPSv2 — B/14

TIPSv2 (Text-Image Pre-training with Spatial awareness) is a family of contrastive vision-language models that produce spatially rich image features aligned with text embeddings. This is the Base variant with 86M vision params and 110M text params.

| Variant | Vision params | Text params | Embed dim |
|---------|--------------|-------------|-----------|
| B/14 | 86M | 110M | 768 |
| L/14 | 303M | 184M | 1024 |
| SO400m/14 | 412M | 448M | 1152 |
| g/14 | 1.1B | 389M | 1536 |

## Usage

```bash
pip install transformers torch torchvision sentencepiece scikit-learn
```

### Load the model

```python
from transformers import AutoModel

model = AutoModel.from_pretrained("google/tipsv2-b14", trust_remote_code=True)
model.eval()
```

### Encode images

Images should be tensors in `[0, 1]` range (just `ToTensor()`, no ImageNet normalization).

```python
from torchvision import transforms
from PIL import Image
import requests

transform = transforms.Compose([
    transforms.Resize((448, 448)),
    transforms.ToTensor(),
])

url = "https://huggingface.co/spaces/google/tipsv2-gpu-explorer/resolve/main/examples/zeroseg/pascal_context_00049_image.png"
image = Image.open(requests.get(url, stream=True).raw)
pixel_values = transform(image).unsqueeze(0)
out = model.encode_image(pixel_values)

print(out.cls_token.shape)     # (1, 1, 768) — global image embedding
print(out.patch_tokens.shape)  # (1, 1024, 768) — per-patch spatial features
```

### Encode text

```python
text_emb = model.encode_text(["a photo of a bus", "a photo of a dog"])
print(text_emb.shape)  # (2, 768) — one embedding per query
```

### Zero-shot classification

```python
import torch.nn.functional as F

classes = ["bus", "car", "dog", "cat"]
cls = F.normalize(out.cls_token[:, 0, :], dim=-1)
text_emb = F.normalize(model.encode_text(classes), dim=-1)
similarity = cls @ text_emb.T
print(classes[similarity.argmax()])  # bus — predicted class
```

### Zero-shot segmentation

```python
classes = ["bus", "snow", "mountain", "house", "road"]
patch_feats = F.normalize(out.patch_tokens, dim=-1)
text_emb = F.normalize(model.encode_text(classes), dim=-1)
seg_map = (patch_feats @ text_emb.T).reshape(32, 32, len(classes)).argmax(dim=-1)
print(seg_map.shape)  # (32, 32) — per-patch class prediction
```

### Visualize spatial features

```python
import numpy as np
from sklearn.decomposition import PCA

spatial = out.patch_tokens.reshape(1, 32, 32, 768)
feat = spatial[0].detach().numpy().reshape(-1, 768)
rgb = PCA(n_components=3).fit_transform(feat).reshape(32, 32, 3)
print(rgb.shape)  # (32, 32, 3) — PCA of patch features as RGB
```

### GPU inference

```python
model = model.cuda()
out = model.encode_image(pixel_values.cuda())
text_emb = model.encode_text(["a city"])
```

## Model details

- **Architecture**: ViT vision encoder (12 layers) + Transformer text encoder (12 layers)
- **Image preprocessing**: resize to any resolution, convert to `[0, 1]` (no ImageNet normalization)
- **Text preprocessing**: SentencePiece tokenizer, lowercased, max 64 tokens
- **Patch size**: 14x14 pixels

## License

Apache 2.0

## Citation

```bibtex
@inproceedings{cao2026tipsv2,
  title     = {{TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment}},
  author    = {Cao, Bingyi and Chen, Koert and Maninis, Kevis-Kokitsi and Chen, Kaifeng and Karpur, Arjun and Xia, Ye and Dua, Sahil and Dabral, Tanmaya and Han, Guangxing and Han, Bohyung and Ainslie, Joshua and Bewley, Alex and Jacob, Mithun and Wagner, Rene and Ramos, Washington and Choromanski, Krzysztof and Seyedhosseini, Mojtaba and Zhou, Howard and Araujo, Andre},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}
```