Point2Depth: a GAN-based Contrastive Learning Approach for mmWave Point Clouds to Depth Images Transformation - Walter Brescia; Giuseppe Roberto; Vito Racanelli; Saverio Mascolo; Luca De Cicco (31st Mediterranean Conference on Control and Automation (MED), 2023)
The perception of the environment is essential in mobile robotics applications as it enables the proper planning and execution of efficient navigation strategies. Optical sensors offer many advantages, ranging from precision to understandability, but they can be significantly impacted by lighting conditions and the composition of the surroundings. In contrast, millimeter wave (mmWave) radar sensors are not influenced by such adverse condition and are capable of detecting partially or fully obstructed obstacles, resulting in more informative point clouds. However, such point clouds are often sparse and noisy. This work presents Point2Depth, a cross-modal contrastive learning approach based on Conditional Generative Adversarial Networks (cGANs) to transform sparse point clouds from mmWave sensors into depth images, preserving the distance information while producing a more comprehensible representation. An extensive data collection phase was conducted to create a rich multimodal dataset with each information associated with a timestamp and a pose. The experimental results demonstrate that the approach is able to produce accurate depth images, even in challenging environmental conditions.
Keywords: Point cloud compression; Image sensors; Navigation; Transforms; Predictive models; Millimeter wave radar; Sensors; mmWave sensors; point clouds; cGAN; contrastive learning; mobile robotics; contrastive generative adversarial networks; depth images