BREAST CANCER AI ASSISTANT: MULTIMODAL DEEP LEARNING-BASED CLASSIFICATION AND SURVIVAL PREDICTION USING CLINICAL AND MAMMOGRAM DATA

Abstract

Breast cancer remains a leading global health challenge, with survival outcomes strongly influenced by complex clinical, pathological, and radiological factors. Traditional prognostic methods often fail to capture nonlinear interactions within data, limiting predictive accuracy. This study aims to develop and evaluate deep learning model for breast cancer classification and survival prediction using primary clinical and mammogram data from a Nigerian hospital. A total of 18,909 clinical records and 7,595 mammogram images were collected, preprocessed, and augmented to ensure robust model training. An EfficientNetB3-based convolutional neural network with dual classification and regression heads was employed for image-based diagnosis and survival prediction, while decision tree, gradient boosting, and linear regression models were applied for structured clinical data analysis. The model was trained using stratified datasets with 70:15:15 splits for training, validation, and testing, and evaluated using metrics such as accuracy, precision, recall, F1-score, AUC, MAE, RMSE, and R². Results demonstrated strong predictive performance, with the EfficientNetB3 model achieving up to 85.88% validation accuracy in classification, and gradient boosting attaining an R² of 0.9072 in survival prediction. Survival analyses revealed expected trends across cancer stage, tumor grade, and treatment adherence. The developed system was deployed as a web-based application, enhancing accessibility for clinical use. In conclusion, the integration of deep learning with clinical and imaging data provides accurate, interpretable, and practical tools for breast cancer prognosis, while future work should incorporate additional multimodal data, improve model sensitivity, and validate findings across larger, diverse populations.