Moderate resolution remote sensing scene classification method based on multi-scale feature fusion

doi:10.3969/j.issn.1000-1565.2019.06.015

Abstract

Abstract: In the scene classification of moderate resolution remote sensing images, the traditional feature extraction method cannot be used to extract comprehensive features, while in the case of scene classification using convolutional neural network, the convolution kernel of the same size cannot extract features in ground objects of different sizes, which reduces the classification accuracy. In order to extract features in different sizes and improve the accuracy of classification, this paper proposes a moderate resolution remote sensing scene classification algorithm based on multi-scale feature fusion. The traditional convolutional neural network was improved to adapt to the moderate resolution remote sensing data set, and multi-scale pooling was added, and multi-level feature maps were input to the full-connection layer for classification. The experiment shows that the feature information extracted by multi-layer feature fusion method is more comprehensive and the classification effect is better than that by single-layer multi-scale pooling method. Compared with other traditional classification methods, our method yields better classification results.- DOI:10.3969/j.issn.1000-1565.2019.06.015基于多尺度特征融合的中分辨率遥感场景分类算法张军^1,2,张敏^1,2,郝小可^1,2,解鹏^1,2(1.河北工业大学人工智能与数据科学学院,天津 300401;2.河北省大数据计算重点实验室,天津 300401)摘要:在对中分辨率遥感图像进行场景分类时,传统的特征提取方法无法提取全面的特征,若使用卷积神经网络进行场景分类,同一大小的卷积核无法提取尺寸大小各异的地物特征,导致分类精度降低.为了提取不同尺寸的地物特征,提高分类精度,本文提出一种基于多尺度特征融合的中分辨率遥感场景分类算法.对传统的卷积神经网络进行改进以适应中分辨率遥感数据集,并在其基础上添加多尺度池化,将连接多层次的特征图谱输入到全连接层进行分类.实验表明,多层特征融合方法提取的特征信息比单层多尺度池化方法提取的特征信息更全面,分类效果更优.与其他的传统分类方法相比,本文方法获得更好的分类结果.关键词:卷积神经网络;多层特征融合;多尺度池化;遥感图像场景分类中图分类号:TP399 文献标志码:A 文章编号:1000-1565(2019)06-0666-07Moderate resolution remote sensing scene classification method based on multi-scale feature fusionZHANG Jun^1,2, ZHANG Min^1,2, HAO Xiaoke^1,2, XIE Peng^1,2(1. School of Artificial Intelligence, Hebei University of Technology, Tianjin 300401, China; 2. Hebei Province Key Laboratory of Big Data Calculation, Tianjin 300401, China)Abstract: In the scene classification of moderate resolution remote sensing images, the traditional feature extraction method cannot be used to extract comprehensive features, while in the case of scene classification using convolutional neural network, the convolution kernel of the same size cannot extract features in ground objects of different sizes, which reduces the classification accuracy. In order to extract features in different sizes and improve the accuracy of classification, this paper proposes a moderate resolution remote sensing scene classification algorithm based on multi-scale feature fusion. The traditional convolutional neural network was improved to adapt to the moderate resolution remote sensing data set, and multi-scale pooling was added, and multi-level feature maps were input to the full-connection layer for classification. The experiment shows that the feature information extracted by multi-layer feature fusion method is more comprehensive and the classification effect is better than that by single-layer multi-scale pooling method. Compared with other traditional classification methods, our method yields better classification results.- 收稿日期:2019-02-15 基金项目:国家自然科学青年基金资助项目(61806071);天津市自然科学基金资助项目(16JCYBJC15600);河北省自然科学基金资助项目(F2017202145) 第一作者:张军(1976—),男,河北张家口人,河北工业大学副教授,博士,主要从事智能信息处理研究.E-mail: zhangjun@scse.hebut.edu.cn 通信作者:郝小可(1985—),男,天津市人,河北工业大学讲师,博士,主要从事机器学习、医学图像分析研究.E-mail:haoxiaoke@scse.hebut.edu.cn第6期张军等:基于多尺度特征融合的中分辨率遥感场景分类算法

Key words: convolutional neural network, multi-level feature fusion, multi-scale pooling, remote sensing image scene classification

CLC Number:

TP399

ZHANG Jun, ZHANG Min, HAO Xiaoke, XIE Peng. Moderate resolution remote sensing scene classification method based on multi-scale feature fusion[J]. Journal of Hebei University (Natural Science Edition), 2019, 39(6): 666-672.

References

[1] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. DOI: 10.1023/B:VISI.0000029664.99615.94.
[2] 刘翠响,李敏,张凤林.基于词包模型和SURF局部特征的人脸识别[J].河北大学学报(自然科学版),2017,37(04):411-418.DOI:10.3969/j.issn.1000 1565.2017.04.013.
[3] 张欣,魏熙宁,王兵.基于视觉词袋和三角函数法的动作识别[J].河北大学学报(自然科学版),2018,38(2):204-210.DOI:10.3969/j.issn.1000 1565.2018.02.013.
[4] LAZEBNIK S, SCHMID C, PONCE J. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[C] //2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'06). IEEE, 2006, 2: 2169-2178. DOI: 10.1109/CVPR.2006.68.
[5] PERRONNIN F, SÀNCHEZ J, MENSINK T. Improving the fisher kernel for large-scale image classification[M] //Computer Vision – ECCV 2010, Berlin, Heidelberg: Springer Berlin Heidelberg, 2010: 143-156. DOI:10.1007/978-3-642-15561-1_11.
[6] YAO X, HAN J, CHENG G, et al. Semantic annotation of high-resolution satellite images via weakly supervised learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(6): 3660-3671. DOI: 10.1109/TGRS.2016.2523563.
[7] ZOU Q, Ni L, ZHANG T, et al. Deep learning based feature selection for remote sensing scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(11): 2321-2325. DOI: 10.1109/LGRS.2015.2475299.
[8] 翟俊海,张素芳,郝璞.卷积神经网络及其研究进展[J].河北大学学报(自然科学版),2017,37(06):640-651.DOI:10.3969/j.issn.1000-1565.2017.06.012.
[9] 张晓男, 钟兴, 朱瑞飞, 等. 基于集成卷积神经网络的遥感影像场景分类[J]. 光学学报, 2018, 38(11): 350-360.DOI:10.3788/AOS201838.1128001.
[10] 张洪群,刘雪莹,杨森,等.深度学习的半监督遥感图像检索[J].遥感学报,2017,21(03):406-414.DOI:10.11834/jrs.20176105.
[11] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324. DOI: 10.1109/5.726791.
[12] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. DOI:10.1145/3065386.
[13] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[14] SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[Z].IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Boston, USA, 2015. DOI:10.1109/cvpr.2015.7298594.
[15] HE K M, ZHANG X Y, REN S Q, et al. Spatial Pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. DOI:10.1109/tpami.2015.2389824.
[16] JARRETT K, KAVUKCUOGLU K, LECUN Y. What is the best multi-stage architecture for object recognition?[C] //2009 IEEE 12th International Conference on Computer Vision, IEEE, 2009: 2146-2153. DOI: 10.1109/ICCV.2009.5459469.
[17] SUN Y, WANG X, TANG X. Deep learning face representation from predicting 10,000 classes[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 1891-1898. DOI:10.1109/cvpr.2014.244.
[18] LIU L, SHEN C, VAN DEN HENGEL A. The treasure beneath convolutional layers: Cross-convolutional-layer pooling for image classification[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 4749-4757. DOI: 10.1109/CVPR.2015.7299107.
[19] 李琳. 鄱阳湖区湿地特征提取研究[D]. 赣州: 江西理工大学, 2014.