USCNet: Transformer-Based Multimodal Fusion with Segmentation Guidance for Urolithiasis Classification

Changmiao Wang; Songqi Zhang; Yongquan Zhang; Yifei Wang; Liya Liu; Nannan Li; Xingzhi Li; Jiexin Pan; Yi Jiang; Xiang Wan; Hai Wang; Ahmed Elazab

doi:10.1109/JBHI.2026.3680589

Back

USCNet: Transformer-Based Multimodal Fusion with Segmentation Guidance for Urolithiasis Classification

Journal article

Open access

Peer reviewed

USCNet: Transformer-Based Multimodal Fusion with Segmentation Guidance for Urolithiasis Classification

Changmiao Wang, Songqi Zhang, Yongquan Zhang, Yifei Wang, Liya Liu, Nannan Li, Xingzhi Li, Jiexin Pan, Yi Jiang, Xiang Wan, …

IEEE journal of biomedical and health informatics, Early Access

2026

DOI: https://doi.org/10.1109/JBHI.2026.3680589

PMID: 41931434

Files and links (1)

pdf

Pre-Proof1.15 MBDownload View

Open Access

Abstract

Clinical Information

Communication systems

Computer networks

DICOM

Digital communication

Dynamic Loss

Electronic mail

Internet

Location awareness

Mobile communication

Multimodal Learning

Network architecture

Urolithiasis Analysis

Wide area networks

Kidney stone disease ranks among the most prevalent conditions in urology, and understanding the composition of these stones is essential for creating personalized treatment plans and preventing recurrence. Current methods for analyzing kidney stones depend on post operative specimens, which prevents rapid classification before surgery. To overcome this limitation, we introduce a new approach called the Urinary Stone Segmentation and Classification Network (USCNet). This innovative method allows for precise preoperative classification of kidney stones by integrating Computed Tomography (CT) images with clinical data from Electronic Health Records (EHR). USCNet employs a Transformer-based multimodal fusion framework with CT-EHR attention and segmentation-guided attention modules for accurate classification. Moreover, a dynamic loss function is introduced to effectively balance the dual objectives of segmentation and classification. Experiments on an in-house kidney stone dataset show that USCNet demonstrates outstanding performance across all evaluation metrics, with its classification efficacy significantly surpassing existing mainstream methods. This study presents a promising solution for the precise preoperative classification of kidney stones, offering substantial clinical benefits. The source code has been made publicly available: https://github.com/fancccc/KidneyStoneSC.

Details

Title: USCNet: Transformer-Based Multimodal Fusion with Segmentation Guidance for Urolithiasis Classification
Authors/Creators: Changmiao Wang - Shenzhen Research Institute of Big Data
Songqi Zhang - Zhejiang University of Finance and Economics
Yongquan Zhang - Zhejiang University of Finance and Economics
Yifei Wang - Zhejiang University of Finance and Economics
Liya Liu - Anhui University of Finance and Economics
Nannan Li - Macau University of Science and Technology
Xingzhi Li - Chinese University of Hong Kong, Shenzhen
Jiexin Pan - Chinese University of Hong Kong, Shenzhen
Yi Jiang - Chinese University of Hong Kong, Shenzhen
Xiang Wan - Shenzhen Research Institute of Big Data
Hai Wang - Murdoch University
Ahmed Elazab - Tsinghua–Berkeley Shenzhen Institute
Publication Details: IEEE journal of biomedical and health informatics, Early Access
Publisher: IEEE
Number of pages: 12
Identifiers: 991005877711907891
Murdoch Affiliation: School of Engineering and Energy; Harry Butler Institute
Language: English
Resource Type: Journal article

Metrics

1 Record Views

See more details