This is a repository of the online localization module of:

[ICRA 24] Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion

• A hierarchical visual tracking of SuperPoint features.
• Visual map is compatible with COLMAP format.

Contributers: Jinhao He *, Huaiyang Huang *, Shuyang Zhang, Jianhao Jiao, Chengju Liu and Ming Liu.

{*} for equal contribution

ubuntu 20.04, ros-noetic, cuda-11.7, cudnn-8.5 libgoogle-glog-dev faiss

under a catkin workspace

mkdir src
cd src
git clone https://github.com/David-Willo/APMP.git --recursive

due to lfs size limit, download extra dependencies to the dependency folder

• dependencies:
  • torch_catkin: catkin wrapper for libtorch and torch_tensorrt repo
  • xslam_test_data: model files and sample data download and extract

under catkin workspace

catkin init
catkin config --extend /opt/ros/noetic
catkin config --merge-devel
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release
catkin build sophus
catkin build xslam_visual_localization

• Download demo data and extract to datapath,

  • the map database is compatible with the colmap sparse reconstruction format,
  • feature extraction and database generation can follow hloc demo

• update parameters in runner script (xslam/script/run_apmp.sh) and execute,

update the CMakeList.txt of protobuf_catkin

set(USE_SYSTEM_PROTOBUF "OFF") to compile with given version

update dependencies/glog_catkin/cmake/Modules/FindGlog.cmake

set(GLOG_PREFER_EXPORTED_GLOG_CMAKE_CONFIGURATION False)

or

build & install an old version of glog (e.g. v0.5.0-rc2)

according to issue543

use /script/convert_model to compile your own model file

Our implementation refers to ORB-SLAM2, hloc, SuperPointPretrainedNetwork, maplab, DLIO We really appreciate these open source projects!

• Early Release (2024-07-10)
• Beta Release (TODO)

If our work has helped you, please cite:

@misc{he2024accuratepriorcentricmonocularpositioning,
      title={Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion}, 
      author={Jinhao He and Huaiyang Huang and Shuyang Zhang and Jianhao Jiao and Chengju Liu and Ming Liu},
      year={2024},
      eprint={2407.09091},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2407.09091}, 
}