There are two challenges in automatic segmentation of complex lung tumors(CLT)on computed tomography(CT)images: 1)The class indistinction between tumors and adjacent tissues; 2)Intra-class inconsistencies within tumors. In order to solve these two challenges, the semantic context prior of the relationship between lung tumor and lung is proposed to be incorporated into the segmentation model, so that the model can learn the semantic context features, and the segmentation of CLT can be reconsidered from a macro perspective. The anatomical prior of lung shape is modeled using information entropy. The proposed novel attention module is embedded in the three-classified U-Net network, so as to guide the training process through domain-specific knowledge. In addition, a boundary enhancement auxiliary- DOI:10.3969/j.issn.1000-1565.2024.04.010结合肺先验与协同深监督的肿瘤自动分割王兵1,2,巨梦仪2,杨颖3,张欣4,翟俊海1,2(1.河北省机器学习与计算智能重点实验室,河北 保定 071002;2.河北大学 数学与信息科学学院,河北 保定 071002;3.河北大学附属医院 放射科,河北 保定 071000;4.河北大学 电子信息工程学院,河北 保定 071002)摘 要:计算机断层扫描图像中复杂肺肿瘤(CLT)的自动分割面临2个挑战:1)肿瘤与邻近组织之间的类间不区分;2)肿瘤内的类内不一致性.为了解决这2个问题,提出将肺肿瘤与肺之间关系的语义上下文先验纳入分割模型中,以便于模型学习到语义上下文特征,并从宏观角度重新思考CLT的分割.利用信息熵对肺形状的解剖先验进行建模.在三分类的U-Net网络中嵌入提出的新型注意模块,从而通过特定领域的知识来指导训练过程.另外,设计了一个可以获得肿瘤边界结构图以及保持肿瘤内部特征一致性的边界增强辅助网络.在此基础上,开发了一个协同深度监督网络框架(CLT-ASegNet),该框架利用混合多尺度语义特征融合进一步提高了模型的判别能力和收敛速度.CLT-ASegNet在CLTCTI分割数据集和Lung16数据集上进行了评估.实验结果表明,所提出的CLT-ASegNet可以有效分割肺肿瘤.关键词:注意力机制; 复杂肺肿瘤分割; 语义上下文先验; 协同深度监督中图分类号:TP391.4 文献标志码:A 文章编号:1000-1565(2024)04-0414-10Automatic segmentation of tumors combining lung prior and synergetic deep supervisionWANG Bing1,2, JU Mengyi2, YANG Ying3, ZHANG Xin4, ZHAI Junhai1,2(1. Hebei Key Laboratory of Machine Learning and Computational Intelligence, Baoding 071002, China;2. College of Mathematics and Information Science,Hebei University, Baoding 071002, China;3. Radiology Department, Hebei University Affiliated Hospital, Baoding 071000, China;4. College of Electronic Information Engineering, Hebei University, Baoding 071002, China)Abstract: There are two challenges in automatic segmentation of complex lung tumors(CLT)on computed tomography(CT)images: 1)The class indistinction between tumors and adjacent tissues; 2)Intra-class inconsistencies within tumors. In order to solve these two challenges, the semantic context prior of the relationship between lung tumor and lung is proposed to be incorporated into the segmentation model, so that the model can learn the semantic context features, and the segmentation of CLT can be reconsidered from a macro perspective. The anatomical prior of lung shape is modeled using information entropy. The proposed novel attention module is embedded in the three-classified U-Net network, so as to guide the training process through domain-specific knowledge. In addition, a boundary enhancement auxiliary- 收稿日期:2023-12-06;修回日期:2024-03-19 基金项目:河北省自然科学基金资助项目(F2021201020);河北省自然科学基金青年科学基金资助项目(F2024502006) 第一作者: 王兵(1967—),女,河北大学教授,主要研究方向为机器学习、计算机视觉等.E-mail:wangbing@hbu.edu.cn 通信作者:张欣(1966—),男,河北大学教授,博士,主要研究方向为机器学习、图像处理等.E-mail:zhangxin@hbu.edu.cn第4期王兵等:结合肺先验与协同深监督的肿瘤自动分割河北大学学报(自然科学版) 第44卷network was designed to obtain tumor boundary structure and maintain the consistency of tumor internal features. On this basis, a collaborative deep supervision network framework(CLT-ASegNet)was developed, which further improved the discriminant ability and convergence speed of the model by using hybrid multi-scale semantic feature fusion. CLT-ASegNet was evaluated on CLTCTI segmentation datasets and Lung16 datasets. The experimental results show that the proposed CLT-ASegNet can effectively segment lung tumors.
WANG Bing
,
JU Mengyi
,
YANG Ying
,
ZHANG Xin
,
ZHAI Junhai
. Automatic segmentation of tumors combining lung prior and synergetic deep supervision[J]. Journal of Hebei University(Natural Science Edition), 2024
, 44(4)
: 414
-423
.
DOI: 10.3969/j.issn.1000-1565.2024.04.010
[1] WANG S, ZHOU M, LIU Z Y, et al. Central focused convolutional neural networks: developing a data-driven model for lung nodule segmentation[J]. Med Image Anal, 2017, 40: 172-183. DOI: 10.1016/j.media.2017.06.014.
[2] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C] //International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham: Springer, 2015: 234-241.10.1007/978-3-319-24574-4_28.
[3] CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C] //European Conference on Computer Vision. Cham: Springer, 2018: 833-851.10.1007/978-3-030-01234-2_49.
[4] ZHANG B H, QI S L, WU Y N, et al. Multi-scale segmentation squeeze-and-excitation UNet with conditional random field for segmenting lung tumor from CT images[J]. Comput Methods Programs Biomed, 2022, 222: 106946. DOI: 10.1016/j.cmpb.2022.106946.
[5] NI Y F, XIE Z, ZHENG D Z, et al. Two-stage multitask U-Net construction for pulmonary nodule segmentation and malignancy risk prediction[J]. Quant Imaging Med Surg, 2022, 12(1): 292-309. DOI: 10.21037/qims-21-19.
[6] ZHOU Y Y, LI Z, BAI S, et al., Prior-aware neural network for partially-supervised multi-organ segmentation[C] //2019 IEEE/CVF International Conference on Computer Vision(ICCV), 2019: 10671-10680. DOI: 10.1109/ICCV.2019.01077.
[7] YU H, LI J Q, ZHANG L X, et al. Design of lung nodules segmentation and recognition algorithm based on deep learning[J]. BMC Bioinformatics, 2021, 22(Suppl 5): 314. DOI: 10.1186/s12859-021-04234-0.
[8] OKTAY O, SCHLEMPER J, LE FOLGOC L, et al. Attention U-net: learning where to look for the pancreas[EB/OL]. 2018: arXiv: 1804.03999. http://arxiv.org/abs/1804.03999.
[9] WANG Z K, ZOU Y N, LIU P X. Hybrid dilation and attention residual U-Net for medical image segmentation[J]. Comput Biol Med, 2021, 134: 104449. DOI: 10.1016/j.compbiomed.2021.104449.
[10] LIN H N, LI Z S, YANG Z F, et al. Variance-aware attention U-Net for multi-organ segmentation[J]. Med Phys, 2021, 48(12): 7864-7876. DOI: 10.1002/mp.15322.
[11] HU J F, WANG H, WANG J, et al. SA-Net: a scale-attention network for medical image segmentation[J]. PLoS One, 2021, 16(4): e0247388. DOI: 10.1371/journal.pone.0247388.
[12] ZHOU Z W, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: A nested U-net architecture for medical image segmentation[C] //International Workshop on Deep Learning in Medical Image Analysis, International Workshop on Multimodal Learning for Clinical Decision Support. Cham: Springer, 2018: 3-11.10.1007/978-3-030-00889-5_1.
[13] CAI Y T, WANG Y. MA-Unet: an improved version of Unet based on multi-scale and attention mechanism for medical image segmentation[C] //Proc SPIE 12167, Third International Conference on Electronics and Communication; Network and Computer Technology(ECNCT 2021), 2022, 12167: 205-211. DOI: 10.1117/12.2628519.
[14] AN F P, LIU J E. Medical image segmentation algorithm based on multilayer boundary perception-self attention deep learning model[J]. Multimed Tools Appl, 2021, 80(10): 15017-15039. DOI: 10.1007/s11042-021-10515-w.
[15] ZHANG H, ZHONG X, LI G, et al. BCU-Net: bridging ConvNeXt and U-Net for medical image segmentation[J]. Comput Biol Med, 2023, 159: 106960. DOI: 10.1016/j.compbiomed.2023.106960.
[16] LI, Z H, LI D H, XU C B, et al. TFCNs: A CNN-Transformer Hybrid Network for Medical Image Segmentation[C] //In International Conference on Artificial Neural Networks, Cham: Springer Nature Switzerland, 2022: 781-792. DOI:10.1007/978-3-031-15937-4_65.
[17] TANG P, YANG P L, NIE D, et al. Unified medical image segmentation by learning from uncertainty in an end-to-end manner[J]. Knowl Based Syst, 2022, 241: 108215. DOI: 10.1016/j.knosys.2022.108215.
[18] CLARK K, VENDT B, SMITH K, et al. The cancer imaging archive(TCIA): maintaining and operating a public information repository[J]. J Digit Imag, 2013, 26(6): 1045-1057. DOI: 10.1007/s10278-013-9622-7.
[19] KERVADEC H, BOUCHTIBA J, DESROSIERS C, et al. Boundary loss for highly unbalanced segmentation[J]. Med Image Anal, 2021, 67: 101851. DOI: 10.1016/j.media.2020.101851.
[20] XUE Y, TANG H, QIAO Z, et al. Shape-aware organ segmentation by predicting signed distance maps[J]. Proc AAAI Conf Artif Intell, 2020, 34(7): 12565-12572. DOI: 10.1609/aaai.v34i07.6946.
[21] BERMAN M, TRIKI A R, BLASCHKO M B. The lovasz-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT, IEEE, 2018: 4413-4421. DOI: 10.1109/cvpr.2018.00464.
[22] ARMATO S G 3rd, MCLENNAN G, BIDAUT L, et al. The lung image database consortium(LIDC)and image database resource initiative(IDRI): a completed reference database of lung nodules on CT scans[J]. Med Phys, 2011, 38(2): 915-931. DOI: 10.1118/1.3528204.
[23] CHEN J N, LU Y Y, YU Q H, et al. TransUNet: transformers make strong encoders for medical image segmentation[EB/OL]. 2021: arXiv: 2102.04306. http://arxiv.org/abs/2102.04306. (
