Abstract
Existing MRI multi-contrast reconstruction methods often fail to capturing complex multimodal relationships due to their basic fusion techniques. To address this issue, a novel network, named Hierarchical Interact Migration Attention Multi-Contrast Network (HIMANet), is proposed to effectively integrate the information across multiple modalities based on a hierarchical attention mechanism. Our network comprises two main innovative components: Dual-Stream Target Enhancement Module (DS-TEM) and Three-Stream Reference Feature Fusion Module (TS-RFFM). DS-TEM is designed to interact and migrate the features between target modality and reference modalities at multiple hierarchical levels. It can capture both local and global contextual information, thereby addressing the issue of insufficient utilization of multi-modal information. Meanwhile, TS-RFFM utilizes a multi-scale approach to enhance feature aggregation by incorporating both low-resolution and high-resolution information from reference modalities, which can effectively solve problem of insufficient feature fusion at a single scale. Experiments demonstrate superiority of our method.