A UK retrospective study assessed Brainomix's e-CTA, an AI tool for large vessel occlusion (LVO) detection, using CT angiography (CTA) in 584 patients evaluated at a single NHS Trust. The study assessed the tool’s real-world performance and reliability, comparing AI results to consultant neuroradiologist reviews, which served as the ground truth.
The AI demonstrated 78% sensitivity, 93% specificity, 50% positive predictive value (PPV), and 98% negative predictive value (NPV). False positives (39 cases) were commonly due to incorrect classification of normal vessels as occlusions (21%) or misinterpretation of other pathologies (13%). False negatives (11 cases) frequently involved missed terminal internal carotid artery (ICA) occlusions (55%).
Compared to prior studies in high-prevalence cohorts, e-CTA had lower real-world PPV (0.5 vs. 0.96 in a prior study). However, performance improved in mechanical thrombectomy (MT) cases, where sensitivity increased to 86%.
The study highlights potential clinical risks, particularly for less experienced clinicians, due to AI misclassification and overreliance on its outputs. It emphasizes the need for local validation before widespread implementation, as real-world disease prevalence differs from controlled research settings. The authors stress the importance of AI governance, clinician training, and risk mitigation to ensure safe deployment.
Read full study
Real world clinical experience of using Brainomix e-CTA software in a medium size Acute NHS Trust
British Journal of Radiology (BJR), 2025
Abstract
Objectives
Artificial intelligence (AI) software including Brainomix "e-CTA" which detect large vessel occlusions (LVO) have clinical potential. We hypothesised that in real world use where prevalence is low, its clinical utility may be overstated.
Methods
In this single centre retrospective service evaluation project, data sent to Brainomix from a medium size acute National Health Service (NHS) Trust hospital between 1/3/2022-1/3/2023 was reviewed. 584 intracranial computed tomography angiogram (CTA) datasets were analysed for LVO by e-CTA. The e-CTA output and radiology report were compared to ground truth, defined by a consultant radiologist with fellowship neuroradiology training, with access to subsequent imaging and clinical notes. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated.
Results
Of 584 cases (45% female, mean age 70 ± 16 years), 9% (n = 50) had LVO. e-CTA had a sensitivity of 0.78 (95% CI 0.64-0.88), specificity of 0.93 (0.9-0.95), PPV of 0.5 (0.42-0.58) and NPV of 0.98 (0.96-0.99). e-CTA had an error rate of 9% (52/584). Erroneous cases were categorised into causes for error. Common causes for false positives included incorrect anatomy (21%, 8/39) and other pathology (13%, 5/39), with several uncategorisable cases (39%, 15/39). Common causes for false negatives included LVO within the terminal internal carotid artery (ICA) (55%, 6/11) and uncategorisable (18%, 2/11).
Conclusions
We demonstrated that PPV of e-CTA is poor in consecutive cases in a real-world NHS setting. We advocate for local validation of AI software prior to clinical use.
Advances in knowledge
Common AI errors were due to anatomical misidentification, presence of other pathology, and misidentifying LVO in the terminal ICA.