BI-RADS Category Prediction from Mammography Images and Mammography Radiology Reports Using Deep Learning: A Systematic Review
Abstract
Women's health and mortality are significantly threatened by breast cancer, underscoring the importance of timely detection and treatment. Mammograms are an extremely useful and trustworthy diagnostic tool for early detection and screening of breast cancer. Mammograms based CADe systems have helped doctors in predicting BI-RADS categories and make better decisions and have somewhat reduced diagnostic errors. As deep learning algorithms advance, deep learning-based CADe systems become a practical means of resolving these problems and greatly improving the accuracy. The purpose of this review is to discuss the current techniques that have been developed for BI-RADS category classification in the fields of deep learning and convolutional neural networks. Additionally, the paper demonstrates the progression of models introduced in the past ten years. It also discusses the shortcomings of models proposed in the literature for the prediction of BI-RADS categories from mammography radiology reports and mammography images, in addition to summarizing the current challenges. Lastly, it proposes a novel multi-modal approach to predict the BI-RADS categories from radiology reports and mammography images.
References
Sung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjomataram, I., Jemal, A., & Bray, F. (2021). Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians, 71(3), 209-249.
Xu, S., Liu, Y., Zhang, T., Zheng, J., Lin, W., Cai, J., ... & Li, Z. (2021). The global, regional, and national burden and trends of breast cancer from 1990 to 2019: results from the global burden of disease study 2019. Frontiers in oncology, 11, 689562.
Giordano, S. H. (2018). Breast cancer in men. New England Journal of Medicine, 378(24), 2311-2320.
Anderson, B. O., Ilbawi, A. M., Fidarova, E., Weiderpass, E., Stevens, L., Abdel-Wahab, M., & Mikkelsen, B. (2021). The Global Breast Cancer Initiative: a strategic collaboration to strengthen health care for non-communicable diseases. The Lancet Oncology, 22(5), 578-581.
Lin, L., Yan, L., Liu, Y., Yuan, F., Li, H., & Ni, J. (2019). Incidence and death in 29 cancer groups in 2017 and trend analysis from 1990 to 2017 from the Global Burden of Disease Study. Journal of hematology & oncology, 12, 1-21.
Ataollahi, M. R., Sharifi, J., Paknahad, M. R., & Paknahad, A. (2015). Breast cancer and associated factors: a review. Journal of medicine and life, 8(Spec Iss 4), 6.
Łukasiewicz, S., Czeczelewski, M., Forma, A., Baj, J., Sitarz, R., & Stanisławek, A. (2021). Breast cancer—epidemiology, risk factors, classification, prognostic markers, and current treatment strategies—an updated review. Cancers, 13(17), 4287.
Misra, S., Solomon, N. L., Moffat, F. L., & Koniaris, L. G. (2010). Screening criteria for breast cancer. Advances in surgery, 44(1), 87-100.
Welch, H. G., Prorok, P. C., O’Malley, A. J., & Kramer, B. S. (2016). Breast-cancer tumor size, overdiagnosis, and mammography screening effectiveness. New England Journal of Medicine, 375(15), 1438-1447.
Mercado, C. L. (2014). Bi-rads update. Radiologic Clinics, 52(3), 481-487.
Orel, S. G., Kay, N., Reynolds, C., & Sullivan, D. C. (1999). BI-RADS categorization as a predictor of malignancy. Radiology, 211(3), 845-850.
Strigel, R. M., Burnside, E. S., Elezaby, M., Fowler, A. M., Kelcz, F., Salkowski, L. R., & DeMartini, W. B. (2017). Utility of BI-RADS assessment category 4 subdivisions for screening breast MRI. American Journal of Roentgenology, 208(6), 1392-1399.
Malmartel, A., Tron, A., & Caulliez, S. (2019). Accuracy of clinical breast examination’s abnormalities for breast cancer screening: cross-sectional study. European Journal of Obstetrics & Gynecology and Reproductive Biology, 237, 1-6.
Ciritsis, A., Rossi, C., Eberhard, M., Marcon, M., Becker, A. S., & Boss, A. (2019). Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. European radiology, 29, 5458-5468.
Gastounioti, A., McCarthy, A. M., Pantalone, L., Synnestvedt, M., Kontos, D., & Conant, E. F. (2019). Effect of mammographic screening modality on breast density assessment: digital mammography versus digital breast tomosynthesis. Radiology, 291(2), 320-327.
Kissane, J., Neutze, J. A., & Singh, H. (2020). Breast imaging. Radiology Fundamentals: Introduction to Imaging & Technology, 139-154.
Bick, U., & Diekmann, F. (2007). Digital mammography: what do we and what don’t we know?. European radiology, 17, 1931-1942.
McCaul, K. D., Branstetter, A. D., Schroeder, D. M., & Glasgow, R. E. (1996). What is the relationship between breast cancer risk and mammography screening? A meta-analytic review. Health Psychology, 15(6), 423.
Horsley, R. K., Kling, J. M., Vegunta, S., Lorans, R., & Patel, B. K. (2019). Baseline mammography: what is it and why is it important? A cross-sectional survey of women undergoing screening mammography. Journal of the American College of Radiology, 16(2), 164-169.
Adcock, K. A. (2004). Initiative to improve mammogram interpretation. The Permanente Journal, 8(2), 12.
Hashemi, R. H., Bradley, W. G., & Lisanti, C. J. (2012). MRI: the basics: The Basics. Lippincott Williams & Wilkins.
Morrow, M., Waters, J., & Morris, E. (2011). MRI for breast cancer screening, diagnosis, and treatment. The Lancet, 378(9805), 1804-1811.
Morris, E. A. (2002). Breast cancer imaging with MRI. Radiologic Clinics, 40(3), 443-466.
Mann, R. M., Kuhl, C. K., & Moy, L. (2019). Contrast‐enhanced MRI for breast cancer screening. Journal of Magnetic Resonance Imaging, 50(2), 377-390.
Kopans, D. B. (1999). Breast-cancer screening with ultrasonography. The Lancet, 354(9196), 2096-2097.
Feig, S. (2010). Cost-effectiveness of mammography, MRI, and ultrasonography for breast cancer screening. Radiologic Clinics, 48(5), 879-891.
Gartlehner, G., Thaler, K. J., Chapman, A., Kaminski, A., Berzaczy, D., Van Noord, M. G., & Helbich, T. H. (2013). Adjunct ultrasonography for breast cancer screening in women at average risk: a systematic review. JBI Evidence Implementation, 11(2), 87-93.
Yaffe, M. J., & Jong, R. A. (2016). Adjunctive ultrasonography in breast cancer screening. The Lancet, 387(10016), 313-314.
Gilbert, F. J., Tucker, L., & Young, K. C. (2016). Digital breast tomosynthesis (DBT): a review of the evidence for use as a screening tool. Clinical radiology, 71(2), 141-150.
Skaane, P., Gullien, R., Bjørndal, H., Eben, E. B., Ekseth, U., Haakenaasen, U., ... & Krager, M. (2012). Digital breast tomosynthesis (DBT): initial experience in a clinical setting. Acta radiologica, 53(5), 524-529.
Abdelrahman, L., Al Ghamdi, M., Collado-Mesa, F., & Abdel-Mottaleb, M. (2021). Convolutional neural networks for breast cancer detection in mammography: A survey. Computers in biology and medicine, 131, 104248.
Hassan, N.M., Hamad, S. & Mahar, K. Mammogram breast cancer CAD systems for mass detection and classification: a review. Multimed Tools Appl 81, 20043–20075 (2022).
Raza, S., Goldkamp, A. L., Chikarmane, S. A., & Birdwell, R. L. (2010). US of breast masses categorized as BI-RADS 3, 4, and 5: pictorial review of factors influencing clinical management. Radiographics, 30(5), 1199-1213.
Tan, Y. J., Sim, K. S., & Ting, F. F. (2017, November). Breast cancer detection using convolutional neural networks for mammogram imaging system. In 2017 International Conference on Robotics, Automation and Sciences (ICORAS) (pp. 1-5). IEEE.
Cè, M., Caloro, E., Pellegrino, M. E., Basile, M., Sorce, A., Fazzini, D., ... & Cellina, M. (2022). Artificial intelligence in breast cancer imaging: Risk stratification, lesion detection and classification, treatment planning and prognosis—A narrative review. Exploration of Targeted Anti-tumor Therapy, 3(6), 795.
Sechopoulos, I., Teuwen, J., & Mann, R. (2021, July). Artificial intelligence for breast cancer detection in mammography and digital breast tomosynthesis: State of the art. In Seminars in cancer biology (Vol. 72, pp. 214-225). Academic Press.
Conti, A., Duggento, A., Indovina, I., Guerrisi, M., & Toschi, N. (2021, July). Radiomics in breast cancer classification and prediction. In Seminars in cancer biology (Vol. 72, pp. 238-250). Academic Press.
Valdora, F., Houssami, N., Rossi, F., Calabrese, M., & Tagliafico, A. S. (2018). Rapid review: radiomics and breast cancer. Breast cancer research and treatment, 169, 217-229.
Gao, Y. E., Lin, J., Zhou, Y., & Lin, R. (2023). The application of traditional machine learning and deep learning techniques in mammography: a review. Frontiers in Oncology, 13, 1213045.
Yu, X., Zhou, Q., Wang, S., & Zhang, Y. D. (2022). A systematic survey of deep learning in breast cancer. International Journal of Intelligent Systems, 37(1), 152-216.
Chen, C. (2017). Science mapping: a systematic review of the literature. Journal of data and information science, 2(2), 1-40.
V.M. Rao, D.C. Levin, L. Parker, B. Cavanaugh, A.J. Frangos, J.H. Sunshine, How widely is computer-aided detection used in screening and diagnostic mammography? J. Am. Coll. Radiol. 7 (10) (2010) 802–805.
De Jesus, C., Moseley, T. W., Diaz, V., Vishwanath, V., Jean, S., Elhatw, A., ... & Patel, M. M. (2023). The benefits of screening mammography. Current Breast Cancer Reports, 15(2), 103-107.
A. Kolak, et al., Primary and secondary prevention of breast cancer, Ann. Agric. Environ. Med. 24 (4) (Dec. 2017) 549–553
T. Kooi, et al., Large scale deep learning for computer aided detection of mammographic lesions, Med. Image Anal. 35 (2017) 303–312
S. Singh, V. Kumar, H. K. Verma, and D. Singh, “SVM based system for classification of microcalcifications in digital mammograms,” in 2006 international conference of the ieee engineering in medicine and biology society, 2006, pp. 4747–4750.
W. Chiracharit, Y. Sun, P. Kumhom, K. Chamnongthai, C. Babbs, E.J. Delp, Normal mammogram classification based on a support vector machine utilizing crossed distribution features, in: In the 26th Annual International Conference of the Ieee Engineering in Medicine and Biology Society vol. 1, 2004, pp. 1581–1584.
A. Bria, C. Marrocco, F. Tortorella, Addressing class imbalance in deep learning for small lesion detection on medical images, Comput. Biol. Med. (2020) 103735
Byrne, C., & Spernak, S. (2005). What is breast density?. Breast Cancer Online, 8(10), e51.
E. Sickles, D. Cj, B. Lw, et al., ACR Bi-rads Atlas: Breast Imaging Reporting and Data System, American College of Radiology, Reston, VA, 2013.
Matsuyama, E., Takehara, M., Takahashi, N., & Watanabe, H. (2023). A Breast Density Classification System for Mammography Considering Reliability Issues in Deep Learning. Open Journal of Medical Imaging, 13(3), 63-83.
Liao, T., Li, L., Ouyang, R., Lin, X., Lai, X., Cheng, G., & Ma, J. (2023). Classification of asymmetry in mammography via the DenseNet convolutional neural network. European journal of radiology open, 11, 100502.
P. Henrot, A. Leroux, C. Barlier, P. G´enin, Breast microcalcifications: the lesions in anatomical pathology, Diagnos. Interv. Imag. 95 (2) (Feb. 2014) 141–152.
L. Wilkinson, V. Thomas, N. Sharma, Microcalcification on mammography: approaches to interpretation and biopsy, Br. J. Radiol. 90 (1069) (Jan. 2017), https://doi.org/10.1259/bjr.20160594, 20160594.
C.K. Bent, L.W. Bassett, C.J.D. Orsi, J.W. Sayre, The positive predictive value of BI-RADS microcalcification descriptors and final assessment categories, Am. J. Roentgenol. 194 (5) (May 2010) 1378–1383.
P.C. Stomper, J. Geradts, S.B. Edge, E.G. Levine, Mammographic predictors of the presence and size of invasive carcinomas associated with malignant microcalcification lesions without a mass, Am. J. Roentgenol. 181 (6) (Dec. 2003) 1679–1684.
Banumathy, D., Khalaf, O. I., Romero, C. A. T., Raja, P. V., & Sharma, D. K. (2023). Breast Calcifications and Histopathological Analysis on Tumor Detection by CNN. Comput. Syst. Sci. Eng., 44(1), 595-612.
J.A. Harvey, Unusual breast cancers: useful clues to expanding the differential diagnosis, Radiology 242 (3) (Mar. 2007) 683–694.
Sun, L., Sun, H., Wang, J., Wu, S., Zhao, Y., & Xu, Y. (2021). Breast mass detection in mammography based on image template matching and CNN. Sensors, 21(8), 2855.
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2021). A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems, 33(12), 6999-7019.
Pinaya, W. H. L., Vieira, S., Garcia-Dias, R., & Mechelli, A. (2020). Convolutional neural networks. In Machine learning (pp. 173-191). Academic Press.
Tsai, K. J., Chou, M. C., Li, H. M., Liu, S. T., Hsu, J. H., Yeh, W. C., ... & Hwang, S. H. (2022). A high-performance deep neural network model for BI-RADS classification of screening mammography. Sensors, 22(3), 1160.
Liu, H., Chen, Y., Zhang, Y., Wang, L., Luo, R., Wu, H., ... & Wang, D. (2021). A deep learning model integrating mammography and clinical factors facilitates the malignancy prediction of BI-RADS 4 microcalcifications in breast cancer screening. European Radiology, 31, 5902-5912.
Schönenberger, C., Hejduk, P., Ciritsis, A., Marcon, M., Rossi, C., & Boss, A. (2021). Classification of mammographic breast microcalcifications using a deep convolutional neural network: A BI-RADS–based approach. Investigative Radiology, 56(4), 224-231.
Boroumandzadeh, M., & Parvinnia, E. (2021). Automated classification of BI-RADS in textual mammography reports. Turkish Journal of Electrical Engineering and Computer Sciences, 29(2), 632-647.
Yang, Y., Zhong, Y., Li, J., Feng, J., Gong, C., Yu, Y., ... & Yao, H. (2024). Deep learning combining mammography and ultrasound images to predict the malignancy of BI-RADS US 4A lesions in women with dense breasts: a diagnostic study. International Journal of Surgery, 10-1097.
Sabani, A., Landsmann, A., Hejduk, P., Schmidt, C., Marcon, M., Borkowski, K., ... & Boss, A. (2022). BI-RADS-based classification of mammographic soft tissue opacities using a deep convolutional neural network. Diagnostics, 12(7), 1564.
Nguyen, H. T., Tran, S. B., Nguyen, D. B., Pham, H. H., & Nguyen, H. Q. (2022, July). A novel multi-view deep learning approach for BI-RADS and density assessment of mammograms. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 2144-2148). IEEE.
Vanderheyden, R., & Xie, Y. (2020, August). Mammography image BI-RADS classification using ohplall. In 2020 IEEE Sixth International Conference on Big Data Computing Service and Applications (BigDataService) (pp. 120-127). IEEE.
Hejduk, P., Marcon, M., Unkelbach, J., Ciritsis, A., Rossi, C., Borkowski, K., & Boss, A. (2022). Fully automatic classification of automated breast ultrasound (ABUS) imaging according to BI-RADS using a deep convolutional neural network. European radiology, 32(7), 4868-4878.
Miao, S., Xu, T., Wu, Y., Xie, H., Wang, J., Jing, S., ... & Liu, Y. (2018). Extraction of BI-RADS findings from breast ultrasound reports in Chinese using deep learning approaches. International journal of medical informatics, 119, 17-21.
Gao, H., Bowles, E. J. A., Carrell, D., & Buist, D. S. (2015). Using natural language processing to extract mammographic findings. Journal of biomedical informatics, 54, 77-84.
Bozkurt, S., & Rubin, D. (2014). Automated detection of ambiguity in BI-RADS assessment categories in mammography reports. In Cross-Border Challenges in Informatics with a Focus on Disease Surveillance and Utilising Big Data (pp. 35-39). IOS Press.
Banerjee, I., Bozkurt, S., Alkim, E., Sagreiya, H., Kurian, A. W., & Rubin, D. L. (2019). Automatic inference of BI-RADS final assessment categories from narrative mammography report findings. Journal of biomedical informatics, 92, 103137.
Castro, S. M., Tseytlin, E., Medvedeva, O., Mitchell, K., Visweswaran, S., Bekhuis, T., & Jacobson, R. S. (2017). Automated annotation and classification of BI-RADS assessment from radiology reports. Journal of biomedical informatics, 69, 177-187.
Casey, A., Davidson, E., Poon, M., Dong, H., Duma, D., Grivas, A., ... & Alex, B. (2021). A systematic review of natural language processing applied to radiology reports. BMC medical informatics and decision making, 21(1), 179.
Oztan, O., Garner, J. P., Constantino, J. N., & Parker, K. J. (2020). Neonatal CSF vasopressin concentration predicts later medical record diagnoses of autism spectrum disorder. Proceedings of the National Academy of Sciences, 117(19), 10609-10613.
Luo, X., Gandhi, P., Storey, S., & Huang, K. (2021). A deep language model for symptom extraction from clinical text and its application to extract covid-19 symptoms from social media. IEEE journal of biomedical and health informatics, 26(4), 1737-1748.
Fernandes, M., Sun, H., Jain, A., Alabsi, H. S., Brenner, L. N., Ye, E., ... & Westover, M. B. (2021). Classification of the disposition of patients hospitalized with COVID-19: reading discharge summaries using natural language processing. JMIR Medical Informatics, 9(2), e25457.
Elmore, J. G., Wells, C. K., Lee, C. H., Howard, D. H., & Feinstein, A. R. (1994). Variability in radiologists' interpretations of mammograms. New England Journal of Medicine, 331(22), 1493-1499.
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., & Chou, K. & Dean, J.(2019). A guide to deep learning in healthcare. Nature medicine, 25(1), 24.
J. Melnikow, et al., Supplemental screening for breast cancer in women with dense breasts: a systematic review for the us preventive services task force, Ann. Intern. Med. 164 (4) (2016) 268–278.
Becker AS, Marcon M, Ghafoor S, Wurnig MC, Frauenfelder T, Boss A. Deep learning in mammography: diagnostic accuracy of a multipurpose image analysis software in the detection of breast cancer. Invest Radiol 2017;52:434-40.
Ueno Y, Forghani B, Forghani R, Dohan A, Zeng XZ, Chamming’s F, et al. Endometrial carcinoma: MR imaging–based texture model for preoperative risk stratification—a preliminary analysis. Radiology 2017;284:748-57.
Copyright (c) 2024 Ashish Shiwlani, Ahsan Ahmad, Muhammad Umar, Nasrullah Dharejo, Anoosha Tahir, Sheena Shiwlani
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
