According to a new study published in Gastroenterology, an artificial intelligence (AI) system significantly increased the accuracy of endoscopists’ evaluations of diminutive colorectal polyps as adenomatous or hyperplastic. Notably, with AI assistance, novice endoscopists were able to achieve near-expert levels of accuracy without additional training.
To create an AI system capable of increasing the accuracy of polyp characterizations by endoscopists of different skill levels, Jin and colleagues developed convolutional neural networks using narrow-band images (NBIs) of diminutive (≤5 mm) colorectal polyps. The training set (1100 adenomatous polyps and 1050 hyperplastic polyps from 1379 patients) was collected between October 2015 and October 2017 at Seoul National University Hospital.
The AI system was then validated on an additional set of 300 images of 180 adenomatous polyps and 120 hyperplastic polyps that were obtained between January 2018 and May 2019.
Next, the team tested the accuracy of 22 endoscopists of three skill levels (7 novices, 4 experts, and 11 NBI-trained experts) both without and with knowledge of the AI system’s results from the processed images of the validation set.
The primary outcome of the study, the effect of AI assistance on the endoscopists’ accuracy in classifying diminutive colorectal polyps, was assessed using mixed-effect logistic and linear regression analyses.
Using the histologic analysis as the reference standard, the AI system distinguished the polyps (adenomatous vs hyperplastic) with 86.7% accuracy. The overall accuracy of all endoscopists was 82.5%, whereas the average accuracies were 73.8% for novices, 83.8% for experts, and 87.6% for NBI-trained experts.
With knowledge of the results from the AI system, the overall accuracy of the endoscopists significantly increased to 88.5% (P <.05), and the average accuracy improved to 85.6% for novices, 89.0% for experts, and 90.2% for NBI-trained experts.
Without AI assistance, the novices had significantly lower accuracy than the experts (P =.049) and the NBI-trained experts (P =.001). With AI assistance, the novices were no longer less accurate than the expert group (both assisted; P =.102). However, the NBI-trained experts maintained significantly higher accuracy compared with the novice group (both assisted; P =.008).
Only 1 endoscopist, an NBI-trained expert, was more accurate than the AI system (92.7%; P =.011).
The AI system also significantly reduced the endoscopists’ time of diagnosis by an average of 0.55 seconds per polyp (from 3.92 to 3.37 seconds per polyp, P =.042).
The investigators acknowledged several limitations of the study. The AI system was trained using high-quality images, which may not always be obtained in the clinical setting. Manual cropping and resizing of images were also conducted before AI training. Only tubular adenoma with low-grade and hyperplastic polyps were included in the study; other polyp types were excluded (eg, serrated lesions, inflammatory polyps, or lymphoid follicles). Issues surrounding the determination of uncertainty with neural networks are still unresolved.
The authors concluded, “by reducing the diagnostic-capability differences between physicians, pathological examinations can be replaced by accurate optical diagnoses with AI assistance that can contribute to significant reductions of medical costs.”
Jin EH, Lee D, Bae JH, et al. Improved accuracy in optical diagnosis of colorectal polyps using convolutional neural networks with visual explanations [published online February 28, 2020]. Gastroenterology. doi: 10.1053/j.gastro.2020.02.036