A Multi-Modal Image Fusion Approach for Visual and Infrared Images via Shearlet-Based Decomposition

Abstract

The integration of optical lens technologies with night vision has become necessary due to the increasing demand to enhance public safety and surveillance, particularly in vulnerable areas, public transportation, and airports. Due to vision clarity issues or the lack of thermal information, conventional single-modality systems often fail to detect concealed dangers. This research presents a multimodal image fusion architecture that integrates visible and infrared (IR) images to enhance hidden weapon detection, thereby mitigating these limitations. Whereas infrared images provide valuable heat signals that can penetrate clothing and reveal hidden objects depending on temperature gradients, visible photographs provide accurate spatial and textural information. In this article, an efficient VR-IR image integration model is proposed by merging distinct images acquired from different sensors:  Visible Images containing high spatial are merged with infrared images containing high thermal radiation information and low spatial resolution details. The proposed fusion algorithm harnesses the attributes of the Shearlet transform (NSST) and spectral residual details information. Furthermore, the proposed architecture yields improved visual and objective results compared to other fusion algorithms. The proposed method surpasses all current methods with the highest fusion rate of 0.9276, minimum information loss (0.0536), and shows artifact (0.0128), indicating nearly no extra noise or visual distortion.