---
title: UniMate-Weights
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/UniMate-Weights"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/UniMate-Weights.md"
repository: AI-ModelScope/UniMate-Weights
last_updated: 2026-10-13
license: mit
base_model:
  - none
base_model_relation: finetune
parameters: 74.1M
tensor_type:
  - F32
library_name:
  - pytorch
  - safetensors
frameworks:
  - pytorch
downloads: 162
stars: 1
tags:
  - motion-generation
  - character-animation
  - skeleton
  - flow-matching
  - diffusion-transformer
  - unimate
---

# UniMate-Weights

> UniMate-Weights - AI-ModelScope 在 ModelScope 开源的模型。UniMate uniml3d60framesgraphadaln — independently trained checkpoint (v2)

AI-ModelScope/UniMate-Weights 是 ModelScope 魔搭社区上的 74.1M 参数机器学习模型，采用 mit 许可，基于 none 构建。

- **Repository**: AI-ModelScope/UniMate-Weights
- **License**: mit
- **Parameters**: 74.1M
- **Base model**: none
- **Tags**: motion-generation, character-animation, skeleton, flow-matching, diffusion-transformer, unimate
- **Downloads**: 162
- **Stars**: 1
- **Last updated**: 2026-10-13

Source: https://www.modelscope.cn/models/AI-ModelScope/UniMate-Weights

---

# UniMate `uniml3d_60frames_graph_adaln` — independently trained checkpoint (v2)

A from-scratch training run of the main [UniMate](https://github.com/Friedrich-M/UniMate) model
("UniMate: One Unified Model to Animate Diverse Skeletons", Mou et al., SIGGRAPH Asia 2026,
[arXiv 2609.05415](https://arxiv.org/abs/2609.05415)) on a single GPU, using the authors' released
code and the shipped `configs/uniml3d_60frames_graph_adaln.json` config unchanged.

**This repository holds two runs.** The files at the root are **v2**, trained on an audited copy of
the dataset from which 13 defective clips were removed and 18 repaired (see [Why v2](#why-v2)).
The earlier **v1** run on the unaudited data is kept under [`v1/`](v1/) for comparison. Use v2.

> **This is not the authors' checkpoint.** It was trained separately, by a third party, on one
> workstation GPU. The paper's example launch uses 8 GPUs, which at the same per-GPU batch size
> would see 8× more samples over the same 120k steps. Expect this model to differ from the official
> release and from any other independent run: different hardware, precision, sampling order and
> random seed all change the final weights. Numbers reported in the paper do not apply to this
> checkpoint unless you evaluate it yourself.

## Which file to use

| File | Use it for | Size |
|---|---|---|
| `model_ema.safetensors` | **Inference (v2).** The exponential-moving-average weights, which the UniMate sampler loads by default and which the authors use for all results. | 283 MB |
| `model.safetensors` | The raw (non-EMA) v2 weights at step 120,000. Slightly noisier than the EMA copy; mainly useful for fine-tuning or comparison. | 283 MB |
| `checkpoints/checkpoint_step_120000.pt` | **Resuming or continuing training** from v2 with `--resume`. Model weights, EMA shadow weights, AdamW state, LR-scheduler state and the step counter. | 1.13 GB |
| `config.json` | The resolved v2 run config, including the auto-computed `max_joints=61` / `max_depth=19`. UniMate's inference scripts read this to rebuild the exact architecture. | |
| `dataset_stats.npy` | Per-channel normalization statistics computed from the v2 training data. Required at inference. | |
| `train_config_as_launched.json` | The config exactly as passed to `accelerate launch`. Identical for v1 and v2. | |
| `logs/events.out.tfevents.*` | v2 TensorBoard scalars (loss terms, grad norm, LR, EMA decay every 50 steps). | |
| `training_curves.png` | v2 and v1 loss curves overlaid, below. | |
| `examples/` | Four sample renders from the final v2 checkpoint. | |
| `audit/` | The data audit that motivated v2: report, per-clip scores, skip lists, the scripts. | |
| `v1/` | The complete v1 run in the same layout, plus its own examples. Superseded. | |

### Running inference with the UniMate code

UniMate's sampler expects an *experiment directory* laid out the way training writes it, and this
repository's root mirrors that layout. Download it, then point `--exp_dir` at it. Exclude `v1/` and
`audit/` to keep the download at 1.7 GB:

```bash
git clone https://github.com/Friedrich-M/UniMate && cd UniMate
# set up the environment per the UniMate README, and build dataset/features/ with stage 4 of
# its data pipeline (the sampler reads the target skeleton's T-pose and topology from there)
hf download tarn59/UniMate-Weights --local-dir outputs/uniml3d_60frames_graph_adaln --exclude "v1/*" --exclude "audit/*"

python -m unimate.inference.sample \
    --exp_dir outputs/uniml3d_60frames_graph_adaln \
    --test_cases_json test_cases.json \
    --num_repetitions 3
```

The sampler picks the latest file in `checkpoints/` (here the single step-120000 v2 checkpoint) and
swaps in its EMA weights automatically, so nothing else needs to be passed. To run v1 instead,
download with `--include "v1/*"` and point `--exp_dir` at the `v1/` directory.

To reproduce v2's training data, apply the skip list in `audit/applied_skip_list_objaverse.txt` and
the export-level repairs recorded in `audit/patch_log.json` (script: `audit/patch_poison.py`) before
running UniMate's stage-4 feature extraction. The 18 repaired clips are edits to the stage-1 export
NPZs, not to the published UniML3D data.

### Loading the safetensors directly

The `.pt` checkpoint is the format UniMate's scripts read; the safetensors files are provided as a
smaller, framework-neutral copy of the weights. To load one into the model class:

```python
import torch
from safetensors.torch import load_model
from unimate.configs.schema import MainConfig
from unimate.models.factory import create_model

cfg = MainConfig.from_json("outputs/uniml3d_60frames_graph_adaln/config.json")
model = create_model(cfg.dataset, cfg.model)
load_model(model, "outputs/uniml3d_60frames_graph_adaln/model_ema.safetensors", strict=True)
model.eval()
```

Use `safetensors.torch.load_model`, not `load_file` + `load_state_dict`: the spectral-RoPE encoder is
one module shared by all ten transformer blocks, so `state_dict()` lists it under eleven names. The
safetensors files store it once, under `rope_j.*`, and `load_model` re-ties the aliases.

## Why v2

The v1 run trained cleanly except for four isolated single-step loss spikes, at steps 43,600
(loss 91, pre-clip gradient norm 588), 78,100 (loss 29), 106,900 and 112,900. Each recovered by the
next logged step, gradient clipping bounded the damage and the EMA averaged it out, so v1 is usable.
But every spike was a wasted, clipped update in a bad direction, and the UniMate README warns that
exactly this pattern comes from defective Objaverse rigs and clips.

To find them, every one of the 10,204 training clips was scored with the frozen v1 EMA model under
three draws of noise, timestep and crop, giving a per-clip loss, alongside a model-free score: the
largest normalized feature value in the clip. The distribution is a clean bell that ends by a loss
of 2, and then three clips sit alone at 7.7, 13.8 and **153** (1,140 × the median). Diagnosis of the
top 24 found one defect behind almost all of them: **the root joint travelling many body lengths per
frame** while the limbs animate normally. In canonical units (body diameter 2.0) a normal clip's
root moves 0.007 per frame; the worst moved 6. Captions such as *"walks in place"* on a root that
crossed 5,000 rig units confirmed the root tracks were exported wrong.

| Disposition | Clips | Detail |
|---|---|---|
| **Removed** | 13 | `SantaRunning` and the 13 flying `900_Gacha*` clips of one rig. The motion *is* world-space flight and the captions describe it; no faithful repair exists. Added to the stage-4 skip list. |
| **Repaired: root pinned** | 17 | All 12 clips of rig `361c8d77…` (root broken on every take) plus five single clips captioned as in-place motion. The root's animated position was replaced with a constant in the source export; horizontal only where the caption implies vertical motion. |
| **Repaired: slowed 2×** | 1 | `Crow-Take_Off`: a real take-off, 15–30× faster than any normal clip. Resampled 26 → 51 frames. |
| **Kept** | 10 | Fast but genuine motions (weapon swings, long-limb lashes, Mixamo acrobatics). |

One repaired clip (`videoClass`) was subsequently rejected by stage 4's own discontinuity filter,
which found nine jump frames once the root drift was gone. Net: **10,204 → 10,191 clips**, a 0.13 %
change. Re-scoring the repaired clips against v1 put every one inside the normal feature range.

Full details, per-clip scores for all 10,204 clips, and the scripts are in [`audit/`](audit/).

### v2 vs v1

Same config, seed, precision and hardware; only the data differs.

![training curves](training_curves.png)

| | v2 (audited) | v1 (original) |
|---|---|---|
| Training clips | 10,191 | 10,204 |
| Worst single step after warm-up | loss 1.36 | loss 91.5 |
| Largest pre-clip gradient norm after warm-up | 9.8 | 588 |
| Loss, steps 110k–120k, mean | 0.159 | 0.185 |
| Loss, final 2k steps, median | 0.148 | 0.141 |
| Wall-clock | 28 h 47 min | 28 h 47 min |

The spikes are gone and the baseline is unchanged to within noise, which is the expected effect of a
0.13 % data change. v2 is not measurably better on training loss than v1; it is cleaner in
provenance and free of the clipped bad updates. No quantitative evaluation has been run on either.

## Model

74.1 M parameters. A factored spatio-temporal diffusion transformer: 10 blocks, 512-wide latent,
8 heads, SwiGLU MLP (2048 nominal), RMSNorm with QK-norm, adaLN-Zero conditioning. Spatial attention
runs within each frame across joints with Graphormer-style graph-distance and edge-type biases and a
sign-invariant spectral RoPE over 8 Laplacian eigenvectors; temporal attention runs within each joint
across frames with 1-D RoPE. The caption (`google/flan-t5-base`, mean-pooled) and the pooled T-pose
enter through the adaLN vector. Trained with flow matching (linear interpolant, velocity prediction)
under a masked L2 loss plus a geodesic rotation loss (weight 0.5), with 10 % caption dropout for
classifier-free guidance. See the UniMate repository for the full description.

## Training run (v2)

| | |
|---|---|
| Config | `configs/uniml3d_60frames_graph_adaln.json`, unmodified |
| Data | UniML3D export (`Linzhan/UniML3D`) → audit (above) → UniMate stage-4 features. Truebones ZOO motions rebuilt locally from a licensed copy. |
| Training clips | **10,191** from 5,755 object types: Objaverse 7,295 (5,697 rigs), Mixamo 2,169 (1 rig, 22-joint core), Truebones 727 (57 species) |
| Filtering | rigs outside 5–60 joints dropped (17 Truebones species, 781 Objaverse rigs); 34 Truebones clips skipped at load for a representation-recovery mismatch; `KingCobra-Walk` absent (its FBX needs the authors' one-off mesh-graft repair); 13 clips removed by the audit |
| Eval split | none (`test_split_ratio = 0`) |
| Hardware | 1× NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB), driver 580.173 |
| Software | PyTorch 2.7.1+cu128 (the repo pins 2.5.1+cu124, which has no sm_120 kernels), Accelerate 1.14.0, Transformers 5.16.1 |
| Precision | **fp32** (`--mixed_precision no`), single process, no gradient accumulation |
| Batch / steps | 16 × 120,000 optimizer steps = 1.92 M samples ≈ 188 passes over the training set |
| Optimizer | AdamW, lr 1e-4, betas (0.9, 0.99), weight decay 1e-5, grad-clip 1.0 |
| Schedule | cosine with 3 % warm-up (3,600 steps) to 5 % of peak |
| EMA | decay 0.9999 with warm-up (0.98 at step 0, reaches ≈0.9999 near 90k) |
| Sampler / augmentation | balanced per-object-type sampler (α = 0.5); joint addition, leaf removal, chain pooling, bone-length perturbation |
| Seed | 10 |
| Throughput | 1.16 steps/s (≈0.86 s/step) |
| Wall-clock | 28 h 47 min (2026-09-22 09:36 → 2026-09-23 14:24 local) |
| Peak GPU memory | ≈62 GB |

### Loss (v2)

| Window (steps) | total loss | velocity L2 | geodesic | pre-clip grad norm (mean) | worst step |
|---|---|---|---|---|---|
| 0 – 10k | 0.505 | 0.429 | 0.152 | 4.57 | 2.38 |
| 10k – 20k | 0.264 | 0.205 | 0.117 | 0.72 | 1.09 |
| 20k – 40k | 0.215 | 0.163 | 0.104 | 0.45 | 0.62 |
| 40k – 60k | 0.195 | 0.146 | 0.098 | 0.40 | 0.48 |
| 60k – 80k | 0.184 | 0.138 | 0.091 | 0.40 | 1.36 |
| 80k – 100k | 0.170 | 0.126 | 0.088 | 0.38 | 0.38 |
| 100k – 120k | 0.163 | 0.121 | 0.084 | 0.41 | 0.49 |
| final 2k steps, median | **0.148** | | | | |

No excluded steps: these are the raw windows. The trainer's non-finite-loss guard never fired.

### Example samples (v2)

Four of the visualizations the trainer rendered at the final checkpoint, sampled from the EMA weights
with classifier-free guidance 3.0 and the dopri5 ODE solver. Stick-figure renders via forward
kinematics, prompt in the title; the rigs are training rigs with their training captions. They are
the first four of the six rigs the visualization step drew at random, not a curated set.

| File | Rig | Prompt |
|---|---|---|
| [`examples/mixamo_leaps_up_and_hangs.mp4`](examples/mixamo_leaps_up_and_hangs.mp4) | Mixamo humanoid, 22 joints | "A person leaps up and grabs on, ending in a free hang with both hands." |
| [`examples/truebones_eagle_sits_spreads_wings.mp4`](examples/truebones_eagle_sits_spreads_wings.mp4) | Truebones Eagle | "An animal sits and spreads its wings." |
| [`examples/objaverse_669a7e6e_scuttles_sideways.mp4`](examples/objaverse_669a7e6e_scuttles_sideways.mp4) | Objaverse rig `669a7e6e57bbceb71e7af9ce_fbx`, crab-like | "An object scuttles sideways with legs moving in unison." |
| [`examples/objaverse_7f020c8e_dances_in_place.mp4`](examples/objaverse_7f020c8e_dances_in_place.mp4) | Objaverse rig `7f020c8ef4754ad9a0e6c065227cbd2a`, biped | "An object dances in place with arms and legs moving." |

v1's examples are under `v1/examples/`.

## License and data provenance

The weights, configs, logs, audit files and example renders in this repository are released under the
[MIT License](LICENSE), matching the license of the UniMate code they were produced with. The
training data keeps its sources' terms: Mixamo assets under Adobe's Mixamo terms, Objaverse-XL assets
under each object's own license, and the Truebones ZOO animal motions under Truebones' commercial
terms, which permit royalty-free use including research but forbid redistributing the motion files.
No motion data is included in this repository; only weights derived from it, the per-channel
normalization statistics, the run configs, and per-clip loss scores. Review the source licenses
before using this model commercially.

## Citation

If you use this checkpoint, please cite the original work:

```bibtex
@article{mou2026unimate,
  title   = {UniMate: One Unified Model to Animate Diverse Skeletons},
  author  = {Mou, Linzhan and Lei, Jiahui and Dou, Zhiyang and Cai, Chenyue and Song, Chaoyue and Finkelstein, Adam and Rusinkiewicz, Szymon},
  journal = {arXiv preprint arXiv:2609.05415},
  year    = {2026}
}
```