随着计算机技术的发展,基于深度学习的医学图像自动分割已经成为人工智能辅助医疗的重要研究方向.为弥补现有神经网络结构对信息提取不足而产生的边缘细节丢失问题,构建了一种基于多维度特征提取网络(RDD-UNet)模型,该模型是基于残差UNet和混合损失函数的三维分割网络,以向肝脏肿瘤分割方法提供高精度的脏器分割结果.首先,该网络从原始CT数据的3个轴向提取信息,以长短跳跃连接的组合形式融合多尺度语义特征,保证了层内和层间信息的充分利用.其次,网络中设计了不平衡深度可分离空洞卷积模块,在提升三维网络计算效率的同时,扩大了体素级别的特征感受范围.最后,针对小尺寸分割目标数据不平衡问题提出了混合损失函数,并与深度监督结构相结合,提升了边缘细节的分割效果.该网络模型从体素、轴向和网络层级3个维度上充分提取特征信息,提高了肝脏分割的准确率,在公共数据集LiTS 2017上的Dice分数达到0.965 2,与其他方法相比达到了较高的精度水平.
With the development of computer technology, automatic medical image segmentation based on deep learning has become an important research field of artificial intelligence assisted medicine. Yet, many existing neural network structures could not be able to integrate enough semantic feature information, which led to the loss of marginal details. In order to solve this problem, a multi-dimensional feature extraction network model(RDD-UNet)is proposed. This model is based on 3D residual UNet with multi-loss, which can provide high-precision organ segmentation results for liver tumor segmentation methods. At first, the network extracts information from the three axes of the original CT data, and long and short skip connections are added into the network ensuring the full usage of inter-slice and intra-slice information. In addition, the unbalanced depth-wise separable dilated convolution block is designed to improve the calculation efficiency of the 3D network and expand the receptive field at the voxel level. Finally, a new multi-loss function is proposed to solve the problem of imbalanced data label on small size- DOI:10.3969/j.issn.1000-1565.2021.04.013基于多维度特征提取网络的肝脏图像分割刘蕊1,续欣莹1,谢珺2(1.太原理工大学 电气与动力工程学院,山西 太原 030024;2.太原理工大学 信息与计算机学院,山西 晋中 030600)摘 要:随着计算机技术的发展,基于深度学习的医学图像自动分割已经成为人工智能辅助医疗的重要研究方向.为弥补现有神经网络结构对信息提取不足而产生的边缘细节丢失问题,构建了一种基于多维度特征提取网络(RDD-UNet)模型,该模型是基于残差UNet和混合损失函数的三维分割网络,以向肝脏肿瘤分割方法提供高精度的脏器分割结果.首先,该网络从原始CT数据的3个轴向提取信息,以长短跳跃连接的组合形式融合多尺度语义特征,保证了层内和层间信息的充分利用.其次,网络中设计了不平衡深度可分离空洞卷积模块,在提升三维网络计算效率的同时,扩大了体素级别的特征感受范围.最后,针对小尺寸分割目标数据不平衡问题提出了混合损失函数,并与深度监督结构相结合,提升了边缘细节的分割效果.该网络模型从体素、轴向和网络层级3个维度上充分提取特征信息,提高了肝脏分割的准确率,在公共数据集LiTS 2017上的Dice分数达到0.965 2,与其他方法相比达到了较高的精度水平.关键词:三维肝脏图像分割;残差连接;混合损失函数;深度可分离空洞卷积中图分类号:TP391 文献标志码:A 文章编号:1000-1565(2021)04-0426-10Multi-dimensional feature extraction network for liver image segmentationLIU Rui1,XU Xinying1,XIE Jun2(1.College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024, China;2.College of Information and Computer, Taiyuan University of Technology, Jinzhong 030600, China)Abstract: With the development of computer technology, automatic medical image segmentation based on deep learning has become an important research field of artificial intelligence assisted medicine. Yet, many existing neural network structures could not be able to integrate enough semantic feature information, which led to the loss of marginal details. In order to solve this problem, a multi-dimensional feature extraction network model(RDD-UNet)is proposed. This model is based on 3D residual UNet with multi-loss, which can provide high-precision organ segmentation results for liver tumor segmentation methods. At first, the network extracts information from the three axes of the original CT data, and long and short skip connections are added into the network ensuring the full usage of inter-slice and intra-slice information. In addition, the unbalanced depth-wise separable dilated convolution block is designed to improve the calculation efficiency of the 3D network and expand the receptive field at the voxel level. Finally, a new multi-loss function is proposed to solve the problem of imbalanced data label on small size- 收稿日期:2021-04-07 基金项目:山西省自然科学基金资助项目(201801D121144;201801D221190) 第一作者:刘蕊(1995-),女,河北衡水人,太原理工大学在读硕士研究生,主要从事计算机视觉和医学图像处理研究.E-mail: 137730427@qq.com 通信作者:谢珺(1979-),女,山西太原人,太原理工大学副教授,博士,主要从事粒计算、数据挖掘和智能信息处理研究.E-mail: xiejun@tyut.edu.cn第4期刘蕊等:基于多维度特征提取网络的肝脏图像分割objections,which is combined with deep supervision structure to improve the segmentation effect of edge details. The proposed method can adequately extract feature information from voxel-wise, axis-wise and hierarchy-wise, and improve the accuracy of liver segmentation. This method gets the Dice score of 0.965 2 on the liver segmentation results of LiTS dataset, which achieves a higher accuracy level compared with other methods.
[1] SIEGEL R L, MILLER K D, FUCHS H E, et al. Cancer statistics, 2021[J]. CA: A Cancer J Clin, 2021, 71(1):7-33. DOI:10.3322/caac.21654.
[2] NAJMI VARZANEH F, PANDEY A, ALIYARI GHASABEH M, et al. Prediction of post-TACE necrosis of hepatocellular carcinoma using volumetric enhancement on MRI and volumetric oil deposition on CT, with pathological correlation[J]. Eur Radiol, 2018, 28(7): 3032-3040. DOI:10.1007/s00330-017-5198-9.
[3] LITJENS G, KOOI T, BEJNORDI B E, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60-88. DOI:10.1016/j.media.2017.07.005.
[4] MASSION P P, ANTIC S, ATHER S, et al. Assessing the accuracy of a deep learning method to risk stratify indeterminate pulmonary nodules[J]. Am J Respir Crit Care Med, 2020, 202(2): 241-249. DOI:10.1164/rccm.201903-0505oc.
[5] LI X, JIANG Y C, LI M L, et al. Lightweight attention convolutional neural network for retinal vessel image segmentation[J]. IEEE Trans Ind Informatics, 2021, 17(3): 1958-1967. DOI:10.1109/TII.2020.2993842.
[6] KUMAR A, UPADHYAY N, GHOSAL P, et al. CSNet: a new DeepNet framework for ischemic stroke lesion segmentation[J]. Comput Methods Programs Biomed, 2020, 193: 105524. DOI:10.1016/j.cmpb.2020.105524.
[7] DUTTA J, CHAKRABORTY D, MONDAL D. Multimodal segmentation of brain tumours in volumetric MRI scans of the brain using time-distributed U-net[C] // Comput Intell Pattern Recognit, 2020: 715-725. DOI:10.1007/978-981-13-9042-5_62.
[8] BEN-COHEN A, DIAMANT I, KLANG E, et al. Fully convolutional network for liver segmentation and lesions detection[C] // Deep Learn Data Labeling Med Appl, 2016: 77-85. DOI:10.1007/978-3-319-46976-8_9.
[9] YUAN Y D. Hierarchical convolutional-deconvolutional neural networks for automatic liver and tumor segmentation[EB/OL]. 2017: arXiv: 1710.04540[cs.CV]. https: //arxiv.org/abs/1710.04540.
[10] TIAN J, LI C, SHI Z C, et al. A diagnostic report generator from CT volumes on liver tumor with semi-supervised attention mechanism[C] / Med Image Comput Comput Assist Interv - MICCAI 2018, 2018: 702-710. DOI:10.1007/978-3-030-00934-2_78.
[11] ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: a nested U-net architecture for medical image segmentation[J]. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support(2018), 2018, 11045: 3-11. DOI:10.1007/978-3-030-00889-5_1.
[12] LI S, TSO G K F, HE K J. Bottleneck feature supervised U-Net for pixel-wise liver and tumor segmentation[J]. Expert Syst Appl, 2020, 145: 113131. DOI:10.1016/j.eswa.2019.113131.
[13] HAN X. Automatic liver lesion segmentation using a deep convolutional neural network method[J]. arXiv preprint arXiv:1704.07239, 2017.
[14] YANG H X, SHAN C F, KOLEN A F, et al. Improving catheter segmentation & localization in 3d cardiac ultrasound using direction-fused fcn[C] //2019 IEEE 16th International Symposium on Biomedical Imaging(ISBI 2019), IEEE, 2019: 1122-1126. DOI:10.1109/ISBI.2019.8759420.
[15] YUN J, PARK J, YU D, et al. Improvement of fully automated airway segmentation on volumetric computed tomographic images using a 2.5 dimensional convolutional neural net[J]. Med Image Anal, 2019, 51: 13-20. DOI:10.1016/j.media.2018.10.006.
[16] LI X M, CHEN H, QI X J, et al. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes[J]. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674. DOI:10.1109/TMI.2018.2845918.
[17] ISENSEE F, JÄGER P F, KOHL S A A, et al. Automated design of deep learning methods for biomedical image segmentation[J]. arXiv preprint arXiv:1904.08128, 2019.
[18] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C] // 2017 IEEE International Conference on Computer Vision(ICCV), IEEE, 2017: 2999-3007. DOI:10.1109/ICCV.2017.324.
[19] SUDRE C H, LI W, VERCAUTEREN T, et al. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations[C] // Deep Learn Med Image Anal Multimodal Learn Clin Decis Support, 2017: 240-248. DOI:10.1007/978-3-319-67558-9_28.
[20] WONG K C L, MORADI M, TANG H, et al. 3D segmentation with exponential logarithmic loss for highly unbalanced object sizes[C] // Med Image Comput Comput Assist Interv - MICCAI 2018, 2018: 612-619. DOI:10.1007/978-3-030-00931-1_70.
[21] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C] // Med Image Comput Comput - Assist Interv - MICCAI 2015, 2015: 234-241. DOI:10.1007/978-3-319-24574-4_28.
[22] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), IEEE, 2016: 770-778. DOI:10.1109/CVPR.2016.90.
[23] WANG P Q, CHEN P F, YUAN Y, et al. Understanding convolution for semantic segmentation[C] // 2018 IEEE Winter Conference on App lications of Computer Vision(WACV), IEEE, 2018: 1451-1460. DOI:10.1109/WACV.2018.00163.
[24] MILLETARI F, NAVAB N, AHMADI S A. V-net: fully convolute-ional neural networks for volumetric medical image segmentation[C] // 2016 Fourth International Conference on 3D Vision(3DV), IEEE, 2016: 565-571. DOI:10.1109/3DV.2016.79.
[25] ZHAO H, GALLO O, FROSIO I, et al. Loss functions for image restoration with neural networks[J]. IEEE Trans Comput Imaging, 2017, 3(1): 47-57. DOI:10.1109/TCI.2016.2644865. (
