Deep Learning with Saliency-Guided Attention for Accurate Breast Lesion Characterization

Abstract

Breast cancer is the second leading cause of cancer-related mortality among women, accounting for 12% of cases. Early diagnosis, based on the identification of radiological features, such as masses and microcalcifications in mammograms, is crucial for reducing mortality rates. However, manual interpretation by radiologists is complex and subject to variability, emphasizing the need for automated diagnostic tools to enhance accuracy and efficiency. This research proposes an intelligent breast lesion characterization system using mammography images by combining deep learning feature extraction with machine learning classification techniques. In this approach, deep features are extracted from mammography images using the Convolutional self-Attention Transformer (CoaT) model, which effectively captures complex visual patterns and structural characteristics of breast tissues. The extracted features are then used to train several classification models including Natural Gradient Boosting (NGB), Histogram Gradient Boosting (HGB), Extreme Gradient Boosting (XGB), and the proposed Fast Interpretable Oblique Tree (FIOT) classifier for accurate lesion classification. The system also integrates an explainable AI module to validate mammography images and provide additional diagnostic insights before classification. Experimental results demonstrate that the proposed FIOT classifier achieves superior performance compared to existing models in terms of accuracy, precision, recall, and F-score. A graphical user interface (GUI) is also developed to enable dataset upload, model training, and prediction operations in a user-friendly manner. The proposed system provides an effective computer-aided diagnostic solution that can support radiologists in improving the accuracy and efficiency of breast lesion detection from mammography images.