Using AI for Fully Automated Endoscopic Disease Activity Assessment in Ulcerative Colitis

An AI-powered, fully automated video analysis system may potentially provide endoscopic disease grading in ulcerative colitis that is equal to the scoring of experienced reviewers.

A fully automated video analysis system has the potential to provide endoscopic disease grading in ulcerative colitis (UC) that is similar to the scoring of experienced reviewers, according to a study published in Gastrointestinal Endoscopy.

Endoscopic measurement of mucosal injury is important for assessing disease severity in UC. However, reviewer subjectivity threatens its accuracy and precision. Gastroenterology has seen a rapid emergence of artificial intelligence methods designed to replicate expert endoscopic interpretation. Researchers aimed to pilot a fully automated video analysis system for endoscopic disease grading in UC using a developmental set of high-resolution UC endoscopic videos that were assigned Mayo endoscopic scores provided by 2 experienced reviewers.

Video still image stacks were then annotated for image quality (informativeness) and Mayo endoscopic scores, and models were trained using convolutional neural networks to predict still image informativeness and Mayo endoscopic scores. A template matching grid search was used to estimate whole-video Mayo endoscopic scores provided by human reviewers using predicted still image Mayo endoscopic scores proportions. This automated whole-video Mayo endoscopic scores workflow was then tested using unaltered endoscopic videos from a multicenter UC clinical trial.

The researchers found that the still image informative classifier had excellent performance with a sensitivity of 0.902 and a specificity of 0.870. In high-resolution videos, fully automated methods currently predicted Mayo endoscopic scores in 78% of videos, while in external clinical trial videos, reviewers agreed on Mayo endoscopic scores in 82.8% of videos. Automated and central reviewer scoring agreement occurred in 57.1% of videos but improved to 69.5% when accounting for reviewer disagreement. In addition, automated Mayo endoscopic scores grading of clinical trial videos, which were often low resolution, correctly distinguished remission versus active disease in 83.7% of videos.

Several study limitations were noted, including the “inherently subjective” nature of endoscopic disease severity assessments, potential bias despite the use of adjudicated dataset to define disease severity, and limitations associated with the “ground truth” used to train and judge the automated systems.

“Artificial intelligence has begun to demonstrate expert level judgement using cleaned and curated data, images, and now is beginning to show promise for understanding endoscopic video,” the authors concluded.

Disclosure: Several study authors declared affiliations with the pharmaceutical industry. Please see the original reference for a full list of authors’ disclosures.

Follow @Gastro_Advisor


Yao H, Najarian K, Gryak J, et al. Fully automated endoscopic disease activity assessment in ulcerative colitis [published online August 15, 2020]. Gastrointest Endosc. doi:  10.1016/j.gie.2020.08.011