心脏磁共振成像(cardiac magnetic resonance,CMR)过程中患者误动、异常幅度的呼吸运动、心律失常会造成CMR图像质量下降,为解决现有的CMR图像增强网络需要人为制作配对数据,且图像增强后部分组织纹理细节丢失的问题,提出了基于空频域特征学习的循环一致性生成对抗网络(cycle-consistent generative adversavial network based on spatial-frequency domain feature learning,SFFL-CycleGAN). 研究结果表明,该网络无须人为制作配对数据集,增强后的CMR图像组织纹理细节丰富,在结构相似度(structural similarity,SSIM)和峰值信噪比(peak signal to noise ratio,PSNR)等方面均优于现有的配对训练网络以及原始的CycleGAN网络,图像增强效果好,有效助力病情诊断.
During the process of cardiac magnetic resonance imaging(CMR), patient misoperation, respiratory movement, and arrhythmia can cause a decrease in CMR image quality. To solve the problem of manually creating paired data and losing some tissue texture details after image enhancement in existing CMR image enhancement networks This paper proposes a cycle consistent generative adversarial network based on spatial frequency domain feature learning(SFFL CycleGAN). The research results show that this network does not require manually creating paired datasets, and the enhanced CMR images have rich texture details. It is superior to existing paired training networks and the original CycleGAN network in terms of Structural Similarity(SSIM)and Peak Signal to Noise Ratio(PSNR). The image enhancement effect is good and effectively assists in disease diagnosis.
[1] 《中国心血管健康与疾病报告》编写组.《中国心血管健康与疾病报告2022》要点解读[J].中国心血管杂志, 2023, 28(4): 297-312. DOI:10.3969/j.issn.1007-5410.2023.04.001.
[2] ZHANG Y X, ZHANG W L, ZHANG Q Y, et al. CMR motion artifact correction using generative adversarial nets[EB/OL]. 2019: 1902.11121. https://arxiv.org/abs/1902.11121v1.
[3] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[EB/OL]. 2014: 1406.2661. https://arxiv.org/abs/1406.2661v1.
[4] PARK D, KANG D U, KIM J, et al. Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training[M] //Computer Vision-ECCV 2020. Cham: Springer International Publishing, 2020: 327-343. DOI:10.1007/978-3-030-58539-6_20.
[5] CHO S J, JI S W, HONG J P, et al. Rethinking coarse-to-fine approach in single image deblurring[C] //2021 IEEE/CVF International Conference on Computer Vision(ICCV). October 10-17, 2021, Montreal, QC, Canada. IEEE, 2021: 4621-4630. DOI:10.1109/ICCV48922.2021.00460.
[6] ZAMIR S W, ARORA A, KHAN S, et al. Restormer: efficient transformer for high-resolution image restoration[C] //2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 18-24, 2022, New Orleans, LA, USA. IEEE, 2022: 5718-5729. DOI:10.1109/CVPR52688.2022.00564.
[7] TSAI F J, PENG Y T, TSAI C C, et al. BANet: a blur-aware attention network for dynamic scene deblurring[J]. IEEE Trans Image Process, 2022, 31: 6789-6799. DOI:10.1109/TIP.2022.3216216.
[8] ZHU Y M, ZHENG W, MA Z P. Superpixel conditional generation adversarial network for CMR artifact correction[J]. Image Vis Comput, 2024, 149: 105112. DOI:10.1016/j.imavis.2024.105112.
[9] KUPYN O, BUDZAN V, MYKHAILYCH M, et al. DeblurGAN: blind motion deblurring using conditional adversarial networks[C] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 8183-8192. DOI:10.1109/CVPR.2018.00854.
[10] KUPYN O, MARTYNIUK T, WU J R, et al. DeblurGAN-v2: deblurring(orders-of-magnitude)faster and better[C] //2019 IEEE/CVF International Conference on Computer Vision(ICCV). October 27 - November 2, 2019, Seoul, Korea(South). IEEE, 2019: 8877-8886. DOI:10.1109/ICCV.2019.00897.
[11] LIU Y B, HARIDEVAN A, SCHOFIELD H, et al. Application of ghost-DeblurGAN to fiducial marker detection[C] //2022 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS). October 23-27, 2022, Kyoto, Japan. IEEE, 2022: 6827-6832. DOI:10.1109/IROS47612.2022.9981701.
[12] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C] //2017 IEEE International Conference on Computer Vision(ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017: 2242-2251. DOI:10.1109/ICCV.2017.244.
[13] 刘勇.心血管核磁共振技术原理及应用进展[J].现代医药卫生, 2017, 33(2): 174-176. DOI:10.3969/j.issn.1009-5519.2017.02.004.
[14] 曲源,路青,蒋杰.磁共振成像运动伪影控制方法及技术进展[J].中国医疗设备, 2022, 37(11): 164-169. DOI:10.3969/j.issn.1674-1633.2022.11.035.
[15] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Commun ACM, 2020, 63(11): 139-144. DOI:10.1145/3422622.
[16] AMIRRAJAB S, AL KHALIL Y, PLUIM J, et al. Cardiac MR image segmentation and quality control in the presence of respiratory motion artifacts using simulated data[M] //Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers. Cham: Springer Nature Switzerland, 2022: 466-475. DOI:10.1007/978-3-031-23443-9_44.
[17] MAO X T, LIU Y M, LIU F Z, et al. Intriguing findings of frequency selection for image deblurring[J]. Proc AAAI Conf Artif Intell, 2023, 37(2): 1905-1913. DOI:10.1609/aaai.v37i2.25281.
[18] KONG L S, DONG J X, GE J J, et al. Efficient frequency domain-based transformers for high-quality image deblurring[C] //2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 17-24, 2023, Vancouver, BC, Canada. IEEE, 2023: 5886-5895. DOI:10.1109/CVPR52729.2023.00570. (
