Ancient Chinese character image retrieval based on space and channel attention fusion network

doi:10.3969/j.issn.1000-1565.2021.05.019

Abstract

Abstract: The image retrieval of ancient Chinese characters is an effective auxiliary tool for the research- DOI:10.3969/j.issn.1000-1565.2021.05.019融合空间及通道注意网络的古籍汉字图像检索田学东^1,2,杨琼^1,2,杨芳^1,2(1. 河北大学网络空间安全与计算机学院,河北保定 071002;2. 河北大学智能图文信息处理研究所,河北保定 071002)田学东河北大学教授,博士生导师,中国中文信息学会理事,中国计算机学会高级会员,河北省人工智能学会理事.主要从事文字识别、数学信息检索、中文信息处理等方面的研究工作和计算机专业的教学工作.主持国家自然科学基金项目2项,河北省自然科学基金项目4项,其他省部级项目4项.获河北省科技进步三等奖3项,河北省教育厅高校科技成果一等奖1项,授权国家发明专利2项,发表论文30余篇.主要讲授电路分析基础、电子线路基础、数字电路与逻辑设计、软件工程、微机原理与汇编语言等本科课程和模式识别、文字识别工程、计算语言学等研究生课程.摘要:古籍汉字图像检索是古籍汉字研究的有效工具.然而,古籍汉字字形复杂、书写风格多变的特点导致传统文字图像检索技术在应用于古籍汉字图像时效果欠佳.针对现有方法在古籍汉字图像特征提取时存在的字形结构细节信息和低层视觉特征提取问题,设计了一种融合空间注意力和通道注意力网络高低层特征的古籍汉字图像检索模型.首先,融合空间注意力的低维特征和通道注意力的高维特征,捕捉古籍汉字空间结构间的依赖关系,提取更丰富的古籍汉字语义特征信息;其次,构建inception残差结构模块,丰富古籍汉字图像特征的感受野,使网络模型更易优化,保留足够的古籍汉字细节信息;最后,运用加权交叉熵损失函数,解决数据集中存在的正负样本不平衡问题,增强检索模型的鲁棒性.在上下、左右、包围和独体结构古籍汉字图像数据集上检索实验的MAP(mean average precision)值分别为77.89%、79.89%、78.21%、80.75%,表明了方法的有效性.关键词:古籍汉字;图像检索;空间注意力;通道注意力;加权交叉熵中图分类号:TP391.43 文献标志码:A 文章编号:1000-1565(2021)05-0623-10Ancient Chinese character image retrieval based on space and channel attention fusion networkTIAN Xuedong^1,2, YANG Qiong^1,2, YANG Fang^1,2(1. School of Cyber Security and Computer, Hebei University, Baoding 071002, China;2.Institute of Intelligent Image and Document Information Processing,Hebei University,Baoding 071002,China)Abstract: The image retrieval of ancient Chinese characters is an effective auxiliary tool for the research- 收稿日期:2021-05-07 基金项目:河北省自然科学基金资助项目(F2019201329) 第一作者:田学东(1963—),男,天津人,河北大学教授,博士生导师,主要从事信息检索、模式识别方向研究. E-mail:xuedong_tian@126.com第5期田学东等:融合空间及通道注意网络的古籍汉字图像检索of ancient Chinese characters. However, the complex shape of ancient Chinese characters and the variety of writing styles have led to poor performance of traditional text image retrieval techniques when applied to ancient Chinese character images. Aiming at the problems in extracting the detailed information of the glyph structure and the low-level visual features in the extraction of the image features of ancient Chinese characters in the existing methods, an ancient Chinese character image retrieval model is designed that combines the high and low-level features of the spatial attention and channel attention network. Firstly, this model integrates the low-dimensional features of spatial attention and the high-dimensional features of channel attention, capture the dependence of the spatial structure of ancient Chinese characters, and extract richer semantic feature information of ancient Chinese characters. Secondly, this model constructs the inception residual structure module to enrich the receptive field of the image features of ancient Chinese characters, make the network model easier to optimize, and retains enough detailed information of the ancient Chinese characters. Finally, the weighted cross-entropy loss function is used to solve the imbalance positive and negative samples in the data set and enhance the robustness of the retrieval model. The MAP values of the retrieval experiment on the ancient Chinese character image data sets with up and down, left and right, enclosing and single-body structures are 77.89%, 79.89%, 78.21%, and 80.75%, respectively, indicating the effectiveness of the method.

Key words: ancient Chinese character, image retrieval, spatial attention, channel attention, weighted cross entropy

CLC Number:

TP391.43

TIAN Xuedong, YANG Qiong, YANG Fang. Ancient Chinese character image retrieval based on space and channel attention fusion network[J]. Journal of Hebei University(Natural Science Edition), 2021, 41(5): 623-632.

References

[1] JIN L W, ZHONG Z Y, YANG Z, et al. Applications of deep learning for handwritten Chinese character recognition: a review[J]. Acta Automatica Sinica, 2016, 42(8): 1125-1141. DOI:10.16383/j.aas.2016.c150725.
[2] 祁俊辉. 基于特征向量和笔顺编码的字形相似算法研究[J]. 重庆邮电大学学报(自然科学版), 2019, 31(6): 885-891. DOI:10.3979/j.issn.1673-825X.2019.06.019.
[3] 马海云, 张忠林. 一种多技术融合的古体汉字识别方法研究[J]. 中央民族大学学报(自然科学版), 2018, 27(3):44-47.DOI:10.3969/j.issn.1005-8036.2018.03.008.
[4] BAI Z L, HUO Q. A study on the use of 8-directional features for online handwritten Chinese character recognition[C] //Eighth International Conference on Document Analysis and Recognition, IEEE, 2005: 262-266. DOI:10.1109/ICDAR.2005.34.
[5] LIN M, SONG R. A stroke-segment-mesh(SSM)glyph description method of Chinese characters[J]. Journal of Computer Research and Development, 2010, 47: 318-327.DOI:10.3724/SP.J.1016.2010.02202.
[6] 冉耕, 黄山, 何志辉, 等. 重叠模糊规范化双弹性网格汉字特征提取[J]. 计算机工程与设计, 2016, 37(1): 211-215. DOI:10.16208/j.issn1000-7024.2016.01.040.
[7] GE Y, HUO Q, FENG Z D. Offline recognition of handwritten Chinese characters using Gabor features, CDHMM modeling and MCE training[C] //IEEE International Conference on Acoustics Speech and Signal Processing, 2002: 1053-1056. DOI:10.1109/ICASSP.2002.5743976.
[8] LI F Y, SHEN Q, LI Y, et al. Handwritten Chinese character recognition using fuzzy image alignment[J]. Soft Computing, 2016, 20(8): 2939-2949. DOI:10.1007/s00500-015-1923-y.
[9] MAPARI S, CHAUDHARY N, NAIK S, et al. Usage of fuzzy rule and SOM based model to identify a handwritten chemical symbol or structures[C] //Second IEEE International Conference on Electrical, Computer and Communication Technologies(ICECCT), 2017:1-4. DOI:10.1109/ICECCT.2017.8117842.
[10] 章夏芬, 庄越挺, 鲁伟明, 等. 根据形状相似性的书法内容检索[J]. 计算机辅助设计与图形学学报, 2005, 17(11): 2565-2569. DOI:10.3321/j.issn:1003-9775.2005.11.029.
[11] ZHANG X F, ZHUANG Y T, WU J Q, et al. Hierarchical approximate matching for retrieval of Chinese historical calligraphy character[J]. Journal of Computer Science and Technology, 2007, 22(4): 633-640. DOI:10.1007/s11390-007-9077-8.
[12] 陈颉, 朱福喜. 根据骨架结构相似性的书法内容分层检索[J]. 小型微型计算机系统, 2010, 31(1): 138-142.
[13] 郭利敏, 葛亮, 刘悦如. 卷积神经网络在古籍汉字识别中的应用实践[J]. 图书馆论坛, 2019, 39(10): 142-148. DOI:10.3969/j.issn.1002-1167.2019.10.016.
[14] XIAO X F, JIN L W, YANG Y F, et al. Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition[J]. Pattern Recognition, 2017, 72: 72-81. DOI:10.1016/j.patcog.2017.06.032.
[15] ZHONG Z Y, JIN L W, XIE Z C. High performance offline handwritten Chinese character recognition using GoogLeNet and directional feature maps[C] //13th International Conference on Document Analysis and Recognition(ICDAR), IEEE, 2015: 846-850. DOI:10.1109/ICDAR.2015.7333881.
[16] SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C] //IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2015: 1-9. DOI:10.1109/CVPR.2015.7298594.
[17] LIU C L. Normalization-cooperated gradient feature extraction for handwritten character recognition[J]. IEEE transactions on Pattern Analysis and Machine Intelligence, 2007, 29(8): 1465-1469. DOI:10.1109/TPAMI.2007.1090.
[18] 张秀玲, 周凯旋, 魏其珺, 等. 多通道交叉融合的深度残差网络脱机手写汉字识别[J]. 小型微型计算机系统, 2019, 40(10): 2232-2235. DOI:10.3969/j.issn.1000-1220.2019.10.039.
[19] 陈站, 邱卫根, 张立臣. 基于改进inception的脱机手写汉字识别[J]. 计算机应用研究: 2020, 37(4):1244-1246. DOI:10.19734/j.issn.1001-3695.2018.09.0784.
[20] 李国强, 周贺, 马锴, 等. 特征分组提取融合深度网络手写汉字识别[J]. 计算机工程与应用, 2020, 56(12): 163-168. DOI:10.3778/j.issn.1002-8331.1904-0029.
[21] CHEN L, WANG S, FAN W, et al. Beyond human recognition: a CNN-based framework for handwritten character recognition[C] //Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition(ACPR), IEEE, 2015: 695-699. DOI:10.1109/ACPR.2015.7486592.
[22] 徐清泉. 基于注意力机制的中文识别算法研究[D]. 武汉: 华中科技大学, 2019. DOI:10.27157/d.cnki.ghzku.2019.004112.
[23] YANG X, HE D F, ZHOU Z H, et al. Improving offline handwritten Chinese character recognition by iterative refinement[C] //14th IAPR International Conference on Document Analysis and Recognition(ICDAR), 2017: 5-10. DOI:10.1109/ICDAR.2017.11.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].(2015-04-10)[2021-05-07].https://arxiv.org/abs/1409.1556v6.
[25] WEN Y, ZHANG K, LI Z, et al. A discriminative feature learning approach for deep face recognition[C] //Computer Vision-ECCV 2016, 2016: 499-515. DOI:10.1007/978-3-319-46478-7_31.
[26] SZEGEDY C, LOFFE S, VANHOUCKE V, et al. Inception-v4, Inception-ResNet and the impact of residual connectionson learning[EB/OL].(2016-08-23)[2021-05-07].https://arxiv.org/abs/1602.07261v2.
[27] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C] //IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016: 770-778. DOI:10.1109/CVPR.2016.90.
[28] WANG T Q, YIN F, LIU C L. Radical-based Chinese character recognition via multi-Labeled learning of deep residualnetworks[C] //2017 14th IAPR International Conference on Document Analysis and Recognition(ICDAR), 2017: 579-584. DOI:10.1109/ICDAR.2017.100.
[29] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010. DOI:10.5555/3295222.3295349.
[30] ZHU X, CHENG D, ZHANG Z, et al. An empirical study of spatial attention mechanisms in deep networks[C] //2019 IEEE/CVF International Conference on Computer Vision(ICCV), 2019: 6687-6696. DOI:10.1109/ICCV.2019.00679.
[31] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis andMachine Intelligence, 2020, 42(8): 2011-2023. DOI:10.1109/TPAMI.2019.2913372.
[32] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M] //Computer Vision- ECCV2018. Cham: Springer International Publishing, 2018: 3-19. DOI:10.1007/978-3-030-01234-2_1.
[33] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C] //Proceedings of the 32nd International Conference on Machine Learning, 2015: 448-456.DOI:10.5555/3045118.3045167.
[34] THEODORIDIS S. Stochastic gradient descent[M] //Machine Learning, Amsterdam: Elsevier, 2015: 161-231. DOI:10.1016/b978-0-12-801522-3.00005-7.
[35] KINGMA D, BA J. Adam: a method for stochastic optimization[EB/OL].(2017-01-30)[2021-05-07]. https://arxiv.org/abs/1412.6980v9.
[36] LI X Y, SUN X F, MENG Y X, et al. Dice loss for data-imbalanced NLP tasks[EB/OL].(2020-08-29)[2021-05-07]. https://arxiv.org/abs/1911.02855v3. (