Object detection and classification of polyps based on improved Faster R-CNN

doi:10.3969/j.issn.1000-1565.2023.01.015

Abstract

Abstract: In order to solve the problem of difficult classification of adenomatous polyps and hyperplastic polyps under colorectoscopy, an object detection and polyp classification model based on improved Faster R-CNN was proposed. Polyp images(1582 adenomatous polyp images, 844 hyperplastic polyp images)were image-enhanced in two ways, and 602 images(386 adenomatous images, 216 hyperplastic- DOI:10.3969/j.issn.1000-1565.2023.01.015基于改进的Faster R-CNN的息肉目标检测和分类方法杨昆^1,2,3,原嘉成¹,高聪⁴,孙宇锋¹,路宇飞¹,常世龙¹,薛林雁^1,2,3(1.河北大学质量技术监督学院,河北保定 071002;2.计量仪器与系统国家地方联合工程研究中心,河北保定 071002;3.河北省新能源汽车动力系统轻量化技术创新中心, 河北保定 071002;4.河北远东通信系统工程有限公司,河北石家庄 050200)摘要:为了解决结肠镜下腺瘤性息肉和增生性息肉不易分型的问题,提出一种基于改进的Faster R-CNN的目标检测及息肉分类模型.在数据预处理阶段,对原有的2 426张息肉图像(1 582张腺瘤性息肉图像,844张增生性息肉图像)通过2种方式进行图像增强,并且通过改进的特征提取、边界框回归以及非极大值抑制的网络,用602张图像(386张腺瘤性图像,216张增生性息肉图像)进行测试.通过实验证明,在交并比(IoU)取0.5时,获得了86.8%的平均精度均值,相较于改进之前提升了2.3%.实验结果验证了该模型的潜在临床应用价值.关键词:息肉;目标检测;分类;Faster R-CNN中图分类号:TP192.7 文献标志码:A 文章编号:1000-1565(2023)01-0103-10Object detection and classification of polyps based onimproved Faster R-CNNYANG Kun^1,2,3, YUAN Jiacheng¹, GAO Cong⁴, SUN Yufeng¹, LU Yufei¹,CHANG Shilong¹, XUE Linyan^1,2,3(1.College of Quality and Technical Supervision, Hebei University, Baoding 071002, China; 2. National & Local Joint Engineering Research Center of Metrology Instrument and System, Baoding 071002, China; 3.New Energy Vehicle Power System Lightweight Technology Innovation Center of Hebei Province, Baoding 071002, China; 4.Hebei Far East Communication System Engineering Co., Ltd, Shijiazhuang 050200, China)Abstract: In order to solve the problem of difficult classification of adenomatous polyps and hyperplastic polyps under colorectoscopy, an object detection and polyp classification model based on improved Faster R-CNN was proposed. Polyp images(1582 adenomatous polyp images, 844 hyperplastic polyp images)were image-enhanced in two ways, and 602 images(386 adenomatous images, 216 hyperplastic- 收稿日期:2022-02-12 基金项目:河北省自然科学基金资助项目(A2011201155);河北大学校长科研基金资助项目(XZJJ201914);河北大学多学科交叉研究项目(DXK201914) 第一作者:杨昆(1976—),男,河北保定人,河北大学教授,博士生导师,博士,主要从事生物医学图像处理方向研究.E-mail:yangkun@hbu.edu.cn 通信作者:薛林雁(1981—),女,河北广平人,河北大学副教授,博士,主要从事生物医学图像处理方向研究.E-mail:lyxue@hbu.edu.cn第1期杨昆等:基于改进的Faster R-CNN的息肉目标检测和分类方法polyp images)were tested. It is proved by experiments that when the intersection-union ratio(IoU)is set to be 0.5, an average precision of 86.8% is obtained, which is 2.3% higher than that before the improvement. The experimental results verified the potential clinical application value of this model.

Key words: polyps, object detection, classification, Faster R-CNN

CLC Number:

TP192.7

YANG Kun, YUAN Jiacheng, GAO Cong, SUN Yufeng, LU Yufei,CHANG Shilong, XUE Linyan. Object detection and classification of polyps based on improved Faster R-CNN[J]. Journal of Hebei University(Natural Science Edition), 2023, 43(1): 103-112.

References

[1] SUNG H, FERLAY J, SIEGEL R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. Ca-cancer J Clin, 2021, 71(3): 209-249. DOI:10.3322/caac.21660.
[2] CARBALLAL S, MOREIRA L, BALAGUER F. Serrated polyps and serrated polyposis syndrome[J]. Cirugía Española Engl Ed, 2013, 91(3):141-148. DOI:10.1016/j.cireng.2013.07.002.
[3] WU X, HE X, LI S, et al. Long non-coding RNA ucoo2kmd.1 regulates CD44-dependent cell growth by competing for miR-211-3p in colorectal cancer[J]. PLoS One, 2016, 11(3): e0151287. DOI:10.1371/journal.pone.0151287.
[4] REX D K, KAHI C, O'BRIEN M, et al. The american society for gastrointestinal endoscopy PIVI(preservation and incorporation of valuable endoscopic innovations)on real-time endoscopic assessment of the histology of diminutive colorectal polyps[J]. Gastrointest Endosc, 2011, 73(3): 419-422. DOI:10.1016/j.gie.2011.01.023.
[5] KESWANI R N, LAW R, CIOLINO J D, et al. Adverse events after surgery for nonmalignant colon polyps are common and associated with increased length of stay and costs[J]. Gastrointest Endosc, 2016, 84(2): 296-303.e1. DOI:10.1016/j.gie.2016.01.048.
[6] PATINO-BARRIENTOS S, SIERRA-SOSA D, GARCIA-ZAPIRAIN B, et al. Kudo's classification for colon polyps assessment using a deep learning approach[J]. Appl Sci, 2020, 10(2): 501. DOI:10.3390/app10020501.
[7] SHARMA P, FRYE J, FRIZELLE F. Response to Re: accuracy of visual prediction of pathology of colorectal polyps: how accurate are we?[J]. ANZ J Surg, 2014, 84(10): 797-798. DOI:10.1111/ans.12790.
[8] LADABAUM U, FIORITTO A, MITANI A, et al. Real-time optical biopsy of colon polyps with narrow band imaging in community practice does not yet meet key thresholds for clinical decisions[J].Gastroenterology, 2013, 144(1): 81-91. DOI:10.1053/j.gastro.2012.09.054.
[9] 蒋西然,蒋韬,孙嘉瑶,等.深度学习人工智能技术在医学影像辅助分析中的应用[J].中国医疗设备,2021,36(6):164-171.DOI:10.3969/j.issn.1674-1633.2021.06.040.
[10] KARKANIS S A, IAKOVIDIS D K, MAROULIS D E, et al. Computer-aided tumor detection in endoscopic video using color wavelet features[J]. IEEE Trans Inf Technol Biomed, 2003, 7(3): 141-152. DOI:10.1109/TITB.2003.813794.
[11] AMELING S, WIRTH S, PAULUS D, et al. Texture-based polyp detection in colonoscopy[C] // Bildverarbeitung für die Medizin 2009: Algorithmen - Systeme - Anwendungen, Proceedings des Workshops vom 22. bis 25. März 2009 in Heidelberg. DBLP, 2009. DOI:10.1007/978-3-540-93860-6_70.
[12] HWANG S, OH J, TAVANAPONG W, et al. Polyp detection in colonoscopy video using elliptical shape feature[C] //2007 IEEE International Conference on Image Processing, San Antonio, TX, USA, IEEE,: II-465. DOI:10.1109/ICIP.2007.4379193.7.
[13] NAWARATHNA R, OH J, MUTHUKUDAGE J, et al. Abnormal image detection in endoscopy videos using a filter bank and local binary patterns[J]. Neurocomputing, 2014, 144: 70-91. DOI:10.1016/j.neucom.2014.02.064.
[14] PACAL I, KARABOGA D. A robust real-time deep learning based automatic polyp detection system[J]. Comput Biol Med, 2021, 134: 104519. DOI:10.1016/j.compbiomed.2021.104519.
[15] SILVA J, HISTACE A, ROMAIN O, et al. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer[J]. Int J Comput Assist Radiol Surg, 2014, 9(2): 283-293. DOI:10.1007/s11548-013-0926-3.
[16] KOMEDA Y, HANDA H, WATANABE T, et al. Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: preliminary experience[J].Oncology(Suppl 1), 2017, 93(1): 30-34. DOI:10.1159/000481227.
[17] BYRNE M F, CHAPADOS N, SOUDAN F, et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model[J].Gut, 2019, 68(1): 94-100. DOI:10.1136/gutjnl-2017-314547.
[18] MESEJO P, PIZARRO D, ABERGEL A, et al. Computer-aided classification of gastrointestinal lesions in regular colonoscopy[J]. IEEE Trans Med Imaging, 2016, 35(9): 2051-2063.DOI:10.1109/TMI.2016.2547947.
[19] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39(6): 1137-1149. DOI:10.1109/TPAMI.2016.2577031.DOI:10.1109/TPAMI.2016.2577031.
[20] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[J]. 2017 IEEE Conf Comput Vis Pattern Recognit CVPR, 2017: 936-944. DOI:10.1109/CVPR.2017.106.
[21] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014. DOI:10.48550/arXiv.1409.1556.
[22] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C] //2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, IEEE,: 770-778. DOI:10.1109/CVPR.2016.90.
[23] ZHANG H, WU C R, ZHANG Z Y, et al. ResNeSt: split-attention networks[C] //2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). New Orleans, LA, USA, IEEE: 2735-2745. DOI:10.1109/CVPRW56347.2022.00309.
[24] WANG J Q, ZHANG W W, CAO Y H, et al. Side-aware boundary localization for more precise object detection[C] //Comput Vis-ECCV 2020, 2020: 403-419. DOI:10.1007/978-3-030-58548-8_24.
[25] BODLA N, SINGH B, CHELLAPPA R, et al. Soft-NMS—improving object detection with one line of code[C] //2017 IEEE International Conference on Computer Vision, Venice, Italy, IEEE: 5562-5570. DOI:10.1109/ICCV.2017.593.
[26] XIE S N, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA, IEEE,: 5987-5995. DOI:10.1109/CVPR.2017.634.
[27] BERNAL J, SÁNCHEZ F J, FERNÁNDEZ-ESPARRACH G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians[J]. Comput Med Imaging Graph, 2015, 43: 99-111. DOI:10.1016/j.compmedimag.2015.02.007.
[28] POGORELOV K, RANDEL K R, GRIWODZ C, et al. KVASIR: A multi-class image dataset for computer aided gastrointestinal disease detection[C] //Proceedings of the 8th ACM on Multimedia Systems Conference. Taipei Taiwan. New York, NY, USA: ACM, 2017: 164–169. DOI:10.1145/3083187.3083212. (