Deep Learning Technology Predicts Inflammatory Activity in Ulcerative Colitis

Researchers developed a deep learning-based computer-assisted diagnosis system for scoring endoscopic images and videos of patients with ulcerative colitis.

Researchers have developed a deep learning-based computer-assisted diagnosis system that can help predict inflammatory activity in patients with ulcerative colitis (UC), according to study findings published in Gastrointestinal Endoscopy.

While endoscopy is used in UC evaluations, an endoscopist’s experience and scoring methods may affect diagnostic accuracy. For the current study, researchers sought to develop a scoring system with deep learning technology to achieve consistent scoring of endoscopic images and videos of UC.

From January 2017 to July 2019, researchers collected endoscopic images and videos from 332 patients who were treated for UC and had a total colonoscopy at a hospital in China. Researchers used the images to train the deep learning technology scoring system, which later itself used the images for scoring videos.

Researchers divided the large intestine into 5 weighted segments:  

  • cecum, 20;
  • transverse colon, 20;
  • descending colon, 20;
  • sigmoid colon, 15;
  • rectum, 10.

The weighted scoring model was then applied to the image sequence. The computer-assisted diagnosis system then calculated an AreaScore based on the Modified Mayo Endoscopic Score (MMES).

Primary outcome measures were the accuracy of the artificial intelligence (AI) algorithm for predicting Mayo and Ulcerative Colitis Endoscopic Index of Severity (UCEIS) classification tasks. A total of 5875 clear white-light images were analyzed.

The trained convolutional neural network predicted a high performance for each primary outcome measure, with 86.54% accuracy in the Mayo-scored classification task. The Kappa coefficient, which measured the agreement between endoscopy and AI, was 0.813 (95% CI, 0.782-0.844). The UCEIS task’s accuracy for assessing vascular pattern, erosions plus ulcers, and bleeding was 90.7%, 84.6%, and 77.7%, respectively, with Kappa coefficients of 0.822 (95% CI, 0.788-0.855), 0.784 (95% CI, 0.744-0.823), and 0.702 (95% CI, 0.612-0.793), respectively.

An assessment of 20 full-length endoscopic videos also found that 91.54% of the areas’ scores predicted by the scoring system were distributed in 4 intervals: 852 cases (0, 0.4), 267 cases (0.72, 1.12), 292 cases (3.2, 3.6), and 144 cases (8.0, 8.4), which accounted for 50.1%, 15.7%, 17.2%, and 8.5%, respectively. These results were consistent with the severities classification of the Mayo score by physicians’ global assessment (0, normal; 1, mild; 2, moderate; and 3, severe).

In an analysis of intestinal inflammatory activities simulation in 9 patients with mild, moderate, and severe symptoms, the computer-assisted diagnostic system predicted each bowel segment’s score and depicted it with 2-dimensional colorized bowel images. The scoring system could also predict the severity score of intestinal inflammatory activities, the researchers noted.

Study limitations include the deep learning network’s prediction ability, which was inadequate for several categories; the inclusion of intestinal segments with high proportions of high-severity lesions; and using a scoring system that was not real-time.

“This CAD [computer-assisted diagnostic] system can not only complete reliable Mayo and UCEIS visual classification tasks, but also complete automatic scoring for full-length endoscope videos,” the study authors wrote. “The classification capacity of Mayo and UCEIS scores for single endoscopic images is equivalent to that of clinical experts. At the same time, the objective and visualized evaluation of the UC inflammatory environment of the whole intestine fills the technical gap in this field.”


Fan Y, Mu R, Xu H, et al. A novel deep learning-based computer-aided diagnosis system for predicting inflammatory activity in ulcerative colitis. Gastrointest Endosc. Published online August 16, 2022. doi:10.1016/j.gie.2022.08.015