How the Hell does NeRF Matter for SLAM?         

Benchmarking Implicit Neural Representation and Geometric Rendering in Real-Time RGB-D SLAM

CVPR 2024

Tongyan Hua 1 & Addison Lin Wang 1,2
1 AI Thrust, HKUST(GZ)    2 Dept. of CSE, HKUST   


Abstract

Implicit neural representation (INR), in combination with geometric rendering, has recently been employed in real-time dense RGB-D SLAM. Despite active research endeavors being made, there lacks a unified protocol for fair evaluation, impeding the evolution of this area. In this work, we establish, to our knowledge, the first open-source benchmark framework to evaluate the performance of a wide spectrum of commonly used INRs and rendering functions for mapping and localization. The goal of our benchmark is to 1) gain an intuition of how different INRs and rendering functions impact mapping and localization and 2) establish a unified evaluation protocol w.r.t. the design choices that may impact the mapping and localization. With the framework, we conduct a large suite of experiments, offering various insights in choosing the INRs and geometric rendering functions: for example, the dense feature grid outperforms other INRs (e.g. tri-plane and hash grid), even when geometric and color features are jointly encoded for memory efficiency. To extend the findings into the practical scenario, a hybrid encoding strategy is proposed to bring the best of the accuracy and completion from the grid-based and decomposition-based INRs. We further propose explicit hybrid encoding for high-fidelity dense grid mapping to comply with the RGB-D SLAM system that puts the premise on robustness and computation efficiency.

benchmark_diagrame

The proposed pipeline for NeRF-SLAM benchmark. The asterisk indicates the existing two values for evaluation.





Inspirations

demo1

Efficacy of New SLAM Framework. Comparison of the hierarchical Dense Grid as implemented in our benchmark with the NICE-SLAM.


new-designs

Inspired New Designs.


Results

demo2

Visual demonstration of Hybrid Encodings.


demo3

Visual demonstration of Explicit Hybrid Encodings.





BibTeX


    @InProceedings{nerfslam24hua,
    author    = {Tongyan Hua and Lin Wang},
    title     = {Benchmarking Implicit Neural Representation and Geometric Rendering in Real-Time RGB-D SLAM},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
}