Neural Real-Time Recalibration for Image-based Multi-Camera Systems

Abstract

   Traditionally, multi-camera calibration relies on physical objects and a controlled environment to achieve high-accuracy results, but this cannot be extended to real-time. Recent advancements in deep learning (DL) have enabled image-based camera calibration, offering real-time operation but often sacrificing accuracy for speed. In this thesis, we address this trade-off by proposing a novel approach that leverages DL models for online and real-time multi-camera calibration with high precision. Current DL methods for camera calibration, while fast, often struggle with real images captured in the wild due to varying conditions. Our approach tackles this challenge by introducing a deep learning model designed for online calibration scenarios from images with low inference time. This model adapts to various camera poses efficiently, ensuring robust calibration across diverse viewpoints.

Central to our approach is the introduction of perturbations into the camera parameters, leveraging known initial parameters and 3D fiducial coordinates. This technique allows the model to learn and predict accurate camera parameters even in uncontrolled settings. Extensive experiments demonstrate the effectiveness of our proposed approach, particularly in scenarios requiring real-time calibration with high precision.