Abstract
Deep neural network (DNN) models applied to medical image analysis are highly vulnerable to adversarial attacks, at both the example (input) and feature (model) levels. Ensuring DNN robustness against these adversarial attacks is crucial for accurate diagnostics. However, existing example-level and feature-level defense strategies, including adversarial training and image-level preprocessing, struggle to achieve effective adversarial robustness in medical image analysis. This challenge arises primarily from difficulties in capturing complex texture features in medical images and the inherent risk of changing intrinsic structural information in the input data. To overcome this challenge, we propose a novel medical imaging protector framework named MI-Protector. This framework comprises two defense methods for unimodal learning and one for multimodal fusion learning, addressing both example-level and feature-level vulnerabilities to robustly protect DNNs against adversarial attacks. For unimodal learning, we introduce an example-level defense mechanism using a generative model with a purifier, termed DGMP. The purifier comprises of a trainable neural network and a pre-trained generator from the generative model, which automatically removes a wide variety of adversarial perturbations. For example and feature-level defense mechanism, we propose unimodal attention noise injection mechanism – (UMAN), to protect learning models at the example and feature layers. To protect the multimodal fusion learning network, we propose the multimodal information fusion attention noise (MMIFAN) injection method, which offers protection at the feature layers while the non-learnable UMAN is applied at the example layer. Extensive experiments conducted on 16 datasets across various medical imaging modalities demonstrate that our framework provides superior robustness compared to existing methods against adversarial attacks. Code: https://github.com/misti1203/MI-Protector.