In a recent retrospective study, Transpara was used to investigate the efficiency and accuracy of an artificial intelligence algorithm in detecting interval cancers (IC) within a middle-income country’s national screening program. The study involved a comparison between an IC group of 406 cases and a control group of 446 women. Expert revaluation categorized the mammograms into three categories as negative, with major, and minor findings. Transpara demonstrated a sensitivity 53.7% and 38.5% for detecting ICs at specificities of 90% and 95%, respectively. It showed higher efficacy for major findings with an area under the curve (AUC) of 0.93, achieving sensitivities of 81.6% and 72.4% at the same specificities. However, at minor findings, the sensitivity dropped to 70% and 53%, with an AUC of 0.87. While Transpara could identify almost half of the obscured or undetectable ICs, the authors advise caution, suggesting additional real-life studies to determine the appropriate approach for utilizing suitable AI algorithms.
Read full study
The efficacy of artificial intelligence (AI) in detecting interval cancers in the national screening program of a middle-income country
Abstract
Objective: We aimed to investigate the efficiency and accuracy of an artificial intelligence (AI) algorithm for detecting interval cancers in a middle-income country’s national screening program.
Material and methods: A total of 2,129,486 mammograms reported as BIRADS 1 and 2 were matched with the national cancer registry for interval cancers (IC). The IC group consisted of 442 cases, of which 36 were excluded due to having mammograms incompatible with the AI system. A control group of 446 women with two negative consequent mammograms was defined as time-proven normal and constituted the normal group. The cancer risk scores of both groups were determined from 1 to 10 with the AI system. The sensitivity and specificity values of the AI system were defined in terms of IC detection. The IC group was divided into subgroups with six-month intervals according to their time from screening to diagnosis: 0–6 months, 6–12 months, 12–18 months, and 18–24 months. The diagnostic performance of the AI system for all patients was evaluated using receiver operating characteristics (ROC) curve analysis. The diagnostic performance of the AI system for major and minor findings that expert readers determined was re-evaluated.
Results: AI labeled 53% of ICs with the highest score of 10. The sensitivity of AI in detecting ICs was 53.7% and 38.5% at specificities of 90% and 95%, respectively. Area under the curve (AUC) of AI in detecting major signs was 0.93 (95% CI: 0.90–0.95) with a sensitivity of 81.6% and 72.4% at specificities of 90% and 95%, respectively (95% CI: 0.73–0.88 and 95% CI: 0.60–0.82 respectively) and minor signs was 0.87 (95% CI: 0.87–0.92) with a sensitivity of 70% and 53% at a specificity of 90% and 95%, respectively (95% CI: 0.65–0.82 and 95% CI: 0.52–0.71 respectively). In subgroup analysis for time to diagnosis, the AUC value of the AI system was higher in the 0–6 month period than in later periods.
Conclusion: This study showed the potential of AI in detecting ICs in initial mammograms and reducing human errors and undetected cancers.
