---
title: ABot-Recon
canonical_url: "https://www.modelscope.cn/studios/amap_cvlab/ABot-Recon"
md_url: "https://www.modelscope.cn/studios/amap_cvlab/ABot-Recon.md"
repository: amap_cvlab/ABot-Recon
chinese_name: "ABot-Recon 在线三维重建"
last_updated: 2026-09-01
sdk_type: gradio
sdk_version: 5.49.1
downloads: 0
stars: 97
---
# ABot-Recon
> ABot-Recon - amap_cvlab 在 ModelScope 创建的在线 Demo。上传单个 RGB 视频，使用 ABot-Recon 进行长序列流式三维重建，输出彩色点云和相机轨迹；支持可选回环优化。
amap_cvlab/ABot-Recon 是 ModelScope 魔搭社区上的在线可交互 Demo（创空间），基于 gradio 5.49.1 构建，中文名为「ABot-Recon 在线三维重建」。
- **Repository**: amap_cvlab/ABot-Recon
- **SDK**: gradio
- **SDK version**: 5.49.1
- **Downloads**: 0
- **Stars**: 97
- **Last updated**: 2026-09-01
Source: https://www.modelscope.cn/studios/amap_cvlab/ABot-Recon
---
# ABot-Recon
### Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction
[English](README.md) | [中文](README_ZH.md)
[](TBD)
[](TBD)
[](TBD)
[](https://huggingface.co/jialidddh/ABot-Recon)
[](TBD)
[](TBD)
[](LICENSE)
> **In one sentence:** ABot-Recon reconstructs long video streams with a fixed 12-frame local context, composing current-frame geometry and adjacent relative poses into a global reconstruction without persistent learned long-range memory.
## Why local context?
Long-horizon streaming reconstruction is often approached by adding increasingly elaborate mechanisms for retaining and fusing long-range state. ABot-Recon takes a deliberately local route. At each time step, it solves the same bounded prediction problem:
- cache KV features from the preceding 11 frames;
- predict a point map $P_i$ in the current camera coordinate system;
- estimate the adjacent relative pose $T_{i-1\leftarrow i}$; and
- recover the global trajectory and point cloud through sequential pose composition.
This design keeps model-state memory and per-frame computation independent of the elapsed sequence length. A lightweight motion-visual rotation refiner and composition-aware pose loss are used to limit drift when local poses are composed over long horizons.
## Results at a glance
| Evaluation | Result | Setting |
|---|---:|---|
| Oxford Spires camera pose | ATE **4.35 m**, RPE-R **0.12°** | Streaming model only; no loop closure |
| Oxford Spires dense reconstruction | CD **1.37 m**, F1 **91.81%** | F1 threshold $\tau=4$ m |
| KITTI-02 streaming efficiency | **24.45 FPS**, **6.71 GiB** | 504×280, NVIDIA H100, input storage excluded |
The full paper reports camera-pose results on KITTI, Oxford Spires, and VBR, together with dense reconstruction on 7Scenes, TUM-Dynamic, and Oxford Spires.
## Installation
The released configuration targets Linux, Python 3.10 or later, PyTorch 2.5.1, and CUDA 12.1. The release environment was validated on NVIDIA A100, while the paper's runtime benchmark uses an NVIDIA H100.
```bash
conda create -n abot-recon python=3.11 -y
conda activate abot-recon
pip install torch==2.5.1 torchvision==0.20.1 \
--index-url https://download.pytorch.org/whl/cu121
pip install -e .
```
### Recommended acceleration
ABot-Recon uses paged KV-cache operators from FlashInfer when they are available and falls back to PyTorch SDPA otherwise. Compiling cuRoPE further accelerates rotary position encoding.
```bash
pip install flashinfer-python
flashinfer show-config
cd abot_recon/modeling/pi3/models/curope
pip install ninja
python setup.py build_ext --inplace
cd -
```
## Model checkpoint
The released checkpoint is hosted at [jialidddh/ABot-Recon](https://huggingface.co/jialidddh/ABot-Recon). The Python API and demo download it automatically and reuse the local Hugging Face cache. For offline inference, download the checkpoint manually and place it at:
```text
checkpoints/abot_recon.safetensors
```
## Quick start
The base model requires neither loop-closure dependencies nor loop assets. Input images are sorted lexicographically, so frame names should be zero-padded (for example, `000001.jpg`, `000002.jpg`, ...).
```bash
python demo.py \
--image-dir examples/images \
--output-dir outputs/demo \
--attention-backend auto \
--no-loop-closure
```
This minimal example disables loop closure and therefore requires only the base installation. It performs one causal pass and writes the raw camera trajectory, adjacent relative poses, local point maps, confidence maps, and run metadata. To enable loop closure, install the optional dependencies and retrieval checkpoints described below, then remove `--no-loop-closure` or replace it with `--loop-closure`.
Useful output controls:
| Option | Effect |
|---|---|
| `--save-world-points` | Transform local point maps using the final trajectory and save a global point cloud |
| `--no-save-local-points` | Skip per-frame local point maps |
| `--no-save-confidence` | Skip confidence maps |
| `--confidence-threshold T` | Mask points below confidence `T` in `[0, 1]` |
| `--loop-closure` / `--no-loop-closure` | Enable or disable optional loop-closure refinement; enabled by default |
| `--start`, `--end`, `--stride` | Select frames from the ordered input stream |
| `--dense-stride N` | Estimate every selected-frame pose but save dense outputs every `N` frames |
| `--max-frames N` | Set the maximum supported stream length; default: `22000` |
### Python API
```python
from pathlib import Path
from abot_recon import ABotRecon
images = sorted(Path("examples/images").glob("*.jpg"))
model = ABotRecon.from_pretrained(
"jialidddh/ABot-Recon",
device="cuda",
attention_backend="auto",
loop_closure=False,
)
result = model.infer(images)
trajectory = result.camera_poses
relative_poses = result.relative_poses
local_points = result.local_points
confidence = result.confidence
```
The checkpoint is downloaded once and then loaded from the Hugging Face cache.
For offline inference, replace the repository ID with a local checkpoint path.
Set `output_world_points=True` to return point maps transformed by the final trajectory. Use `dense_output_indices` when dense geometry is needed for only a subset of frames.
## Optional loop closure
The learned model does not depend on loop closure. When a sequence contains useful revisits, the optional backend retrieves candidate frame pairs with DINOv2-SALAD descriptors, predicts relative-pose constraints with ABot-Recon, and refines the trajectory through sparse pose-graph optimization.
Install the optional dependencies and download the retrieval checkpoints:
```bash
pip install -e ".[loop]"
python scripts/download_loop_assets.py --output-dir checkpoints/loop
```
Expected files:
```text
checkpoints/
├── abot_recon.safetensors
└── loop/
├── dino_salad.ckpt
└── dinov2_vitb14_pretrain.pth
```
Run inference with loop closure:
```bash
abot-recon \
--image-dir examples/images \
--checkpoint checkpoints/abot_recon.safetensors \
--output-dir outputs/demo_loop \
--loop-closure \
--save-world-points
```
When loop closure is enabled, `camera_poses` stores the refined trajectory, while `camera_poses_noloop` preserves the raw streaming prediction.
## Outputs
The exact set of files follows the selected output options:
```text
outputs/demo/
├── camera_poses.npy
├── relative_poses.npy
├── camera_poses_noloop.npy
├── relative_poses_noloop.npy
├── camera_poses_loop.npy # only with loop closure
├── relative_poses_loop.npy # only with loop closure
├── local_points.pt # enabled by default
├── world_points.pt # with --save-world-points
├── colors.pt # RGB aligned with saved point maps
├── confidence.pt # enabled by default
├── confidence_mask.pt # enabled by default
└── metadata.json
```
Local point maps remain in their corresponding camera coordinate systems. World points are generated using the final selected trajectory.
### Visualization
```bash
python scripts/export_reconstruction_ply.py \
--poses outputs/demo/camera_poses.npy \
--points outputs/demo/local_points.pt \
--colors outputs/demo/colors.pt \
--output outputs/demo/reconstruction.ply \
--bev-output outputs/demo/trajectory_bev.png
```
This creates an RGB point-cloud PLY and a separate BEV trajectory PNG.
## Evaluation
Camera-pose and dense-reconstruction protocols are maintained on the `eval` branch:
```bash
git switch eval
```
That branch documents dataset preparation, third-party checkpoints, benchmark commands, and metric aggregation. Dense reconstruction is evaluated without loop closure to match the paper protocol.
## Tests
```bash
pytest -q
```
CUDA-specific and real-checkpoint tests are available separately:
```bash
ABOT_RECON_REQUIRE_CUROPE=1 pytest -q tests/test_curope_parity.py
ABOT_RECON_CHECKPOINT=checkpoints/abot_recon.safetensors \
ABOT_RECON_IMAGE_DIR=examples/images \
ABOT_RECON_DEVICE=cuda \
pytest -q tests/integration/test_real_checkpoint.py
```
## Release status
- [ ] Training code and recipes
- [x] Public model checkpoint
- [x] Inference and evaluation code
## Citation
```bibtex
@article{abot_recon2026,
title = {Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction},
author = {{AMAP CV Lab}},
journal = {arXiv preprint arXiv:TBD},
year = {2026},
eprint = {TBD},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
doi = {TBD},
url = {https://arxiv.org/abs/TBD}
}
```
## License and acknowledgements
Source code is released under the [Apache License 2.0](LICENSE). Model weights are governed by [MODEL_LICENSE.md](MODEL_LICENSE.md), and third-party components are documented in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
ABot-Recon builds on Pi3 and draws inspiration from CroCo, DUSt3R, DINOv2, SALAD, FlashInfer, LingBot-Map, HorizonStream, and LongStream. We thank their authors and contributors.