Improving Real-Time Aerial 3D Reconstruction: Towards Fusion of a Hand-Crafted SfM Algorithm with a Data-Driven Deep Neural Network
Jack Akers, Andrew Buck, Derek Anderson, James Keller, Raub Camaioni, Matthew Deardorff, Robert Luke III
2023 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), St. Louis, MO, USA, 2023
AIPR, Depth Estimation, Simulation, Drones
Abstract
Depth estimation from imagery is a crucial component of numerous real-time systems, such as unmanned aerial vehicles. Traditional approaches, like structure from motion (SfM), rely on stereo cameras or sequential images from a single moving camera to infer depth. However, these methods often yield sparse or incomplete depth maps. Moreover, their accuracy is contingent on factors like baseline and platform motion. Recent advancements leveraging deep neural networks have exhibited remarkable outcomes in predicting dense depth from a single image. Nonetheless, the reliability of these black box models is a concern, and the depth approximations they provide often require refinement. In this study, we delve into the merits and drawbacks of each approach, exploring the potential synergies that could arise from their combination. The initial approach, serving as our baseline, relies solely on sparse SfM depth estimates. In contrast, the second method employs spatial interpolation to fill in missing SfM depths. These methodologies are then contrasted with the performance of two deep nets—one trained on simulated data and the other on real data. The validation of all four approaches is conducted against a gold standard generated using the Unreal Engine.
Files
[paper]