Hierarchical Pose Graphs for Long-Horizon 3D Reconstruction

Reconstruct long image sequences by aligning overlapping chunks and refining camera poses.
Inference with VGGT-Omega or LingBot-MAP, with SALAD loop retrieval.

Demo  ·  Installation  ·  Weights  ·  Input  ·  Inference  ·  Outputs

---

Demo

HiP3R.mp4

Installation

Use Python 3.10 or newer. The demo defaults to an NVIDIA GPU with CUDA and bfloat16 inference.

From the repository root:

conda create -n hip3r python=3.10
conda activate hip3r
pip install -e '.[fast]'

Check that PyTorch can access your GPU:

python -c "import torch; print(torch.cuda.is_available())"

Weights

Download the checkpoints from the corresponding upstream projects and place them in checkpoints/.

Model	Checkpoint	Source
VGGT-Omega	vggt_omega_1b_512.pt	VGGT-Omega
LingBot-MAP	lingbot-map-long.pt	LingBot-MAP
SALAD	dino_salad.ckpt	SALAD
DINOv2 ViT-B/14	dinov2_vitb14_pretrain.pth	DINOv2

checkpoints/
├── vggt_omega_1b_512.pt
├── lingbot-map-long.pt
├── dino_salad.ckpt
└── dinov2_vitb14_pretrain.pth

Download the reconstruction checkpoint for your chosen backend, plus both dino_salad.ckpt and the DINOv2 checkpoint for the required SALAD loop retrieval.

Input

The demo accepts a video or a directory of JPG/PNG frames. For an image sequence, name the files so that alphabetical order matches frame order:

data/example/
├── 000000.png
├── 000001.png
├── 000002.png
└── ...

Video input is decoded into frames under OUTPUT/input_frames/. Use a fresh output directory for each video run.

Inference

Edit the settings in run.sh. Each ${VAR:-value} expression supplies a default that you can also override through an environment variable.

INPUT="${INPUT:-data/example}" # Or data/example.mp4.
OUTPUT="${OUTPUT:-outputs/example}"
BACKEND="${BACKEND:-vggt_omega}"
CHECKPOINT="${CHECKPOINT:-checkpoints/vggt_omega_1b_512.pt}"
DEVICE="${DEVICE:-cuda:0}"
DTYPE="${DTYPE:-bfloat16}"

Run:

bash run.sh

To use LingBot-MAP without editing the script:

BACKEND=lingbot_map CHECKPOINT=checkpoints/lingbot-map-long.pt bash run.sh

You can also pass arguments directly to Python:

python demo.py \
  --input data/example \
  --output outputs/example \
  --backend vggt_omega \
  --checkpoint checkpoints/vggt_omega_1b_512.pt \
  --device cuda:0 \
  --dtype bfloat16

Use python demo.py --help for all options. Relative paths in run.sh are resolved from the repository root.

VGGT-Omega uses 60-frame chunks with 30-frame overlap. LingBot-MAP uses 120-frame chunks with 32-frame overlap. Both use keyframe pose refinement. SALAD loop retrieval runs before reconstruction for both backends.

Outputs

Results are saved to OUTPUT:

File	Contents
camera_poses.txt	One flattened 4×4 camera-to-world matrix per frame
intrinsics.txt	One flattened 3×3 camera intrinsic matrix per frame
chunks/chunk_*.npz	Per-chunk points, confidence, poses, intrinsics, and available depth fields
reconstruction.json	Chunk ranges, alignment transforms, and reconstruction diagnostics

Load the camera poses and chunk geometry with NumPy:

import numpy as np

poses = np.loadtxt("outputs/example/camera_poses.txt").reshape(-1, 4, 4)
with np.load("outputs/example/chunks/chunk_0000.npz") as chunk:
    points = chunk["world_points"]
    confidence = chunk["world_points_conf"]

Acknowledgements

This code uses VGGT-Omega, LingBot-MAP, SALAD, and DINOv2.

License

Original HiP3R code is released under the Apache License 2.0.

Third-party notices are listed in THIRD_PARTY_NOTICES.md.

Citation

@article{xu2026hip3r,
  title={HiP3R: Hierarchical Pose Graphs for Long-Horizon 3D Reconstruction},
  author={Xu, Zunnan and Zhou, Shangchen and Luo, Yihang and Loy, Chen Change},
  journal={arXiv preprint arXiv:2610.31635},
  year={2026}
}