基于目标检测和迁移时间序列的教室人员检测

doi:10.3969/j.issn.1000-1565.2023.04.012

摘要/Abstract

摘要： 为了解决教育智能化重构中对于教室人员的高效监督管理和数据分析,结合单阶段目标检测算法的优良特性和卷积神经网络(convolutional neural networks,CNN)良好的特征提取能力,提出了一种基于注意力机制网络和迁移时间序列改进YOLO目标检测算法的教室人员目标检测算法.首先,对源视频流进行逐帧抽取和非畸变的图像放缩,通过生成对抗网络(generative adversarial network,GAN)进行图像超分辨处理;其次,对每帧图像进行多尺度采样和初步目标检测;然后,根据不同尺度得到的候选结果进行非极大抑制(non maximum suppression,NMS)以去除置信度较低的个体;之后,对候选结果进行融合,再使用交并比(intersection over union,IoU)进行重叠度计算以更新数据、去除重合或过于紧密的定位位置,然后将当前帧的检测结果与先前时间区间中的检测结果作为时间序列进行统计学数据迁移融合(time series migration,TSM)获得最后的检测结果.实验结果表明,本文方法不仅有效地提升了教室人员目标检测的准确率,并且可以进行实时检测.

关键词: 目标检测, 多尺度图像特征, 超分辨率, 注意力机制网络, 迁移时间序列

Abstract: In order to solve the problem of efficient supervision and management and data analysis of classroom persons in the intelligent reconstruction of education, this paper combines the good characteristics of single-stage object detection algorithm and the good special extraction ability of convolutional neural networks(CNN), and proposes an improved YOLO object detection algorithm based on attention network and time series migration. Firstly, this paper performs frame-by-frame extraction and non-distortion image deflation on the source video stream, and performs image super-resolution processing- DOI:10.3969/j.issn.1000-1565.2023.04.012基于目标检测和迁移时间序列的教室人员检测丛帅¹,杨磊²,华征豪³,杨晓晖²(1.河北大学工商学院,河北保定 071000;2.河北大学教务处,河北保定 071002;3.河北大学网络空间安全与计算机学院,河北保定 071000)摘要:为了解决教育智能化重构中对于教室人员的高效监督管理和数据分析,结合单阶段目标检测算法的优良特性和卷积神经网络(convolutional neural networks,CNN)良好的特征提取能力,提出了一种基于注意力机制网络和迁移时间序列改进YOLO目标检测算法的教室人员目标检测算法.首先,对源视频流进行逐帧抽取和非畸变的图像放缩,通过生成对抗网络(generative adversarial network,GAN)进行图像超分辨处理;其次,对每帧图像进行多尺度采样和初步目标检测;然后,根据不同尺度得到的候选结果进行非极大抑制(non maximum suppression,NMS)以去除置信度较低的个体;之后,对候选结果进行融合,再使用交并比(intersection over union,IoU)进行重叠度计算以更新数据、去除重合或过于紧密的定位位置,然后将当前帧的检测结果与先前时间区间中的检测结果作为时间序列进行统计学数据迁移融合(time series migration,TSM)获得最后的检测结果.实验结果表明,本文方法不仅有效地提升了教室人员目标检测的准确率,并且可以进行实时检测.关键词:目标检测;多尺度图像特征;超分辨率;注意力机制网络;迁移时间序列中图分类号:TP391.41 文献标志码:A 文章编号:1000-1565(2023)04-0432-10Classroom personnel detection based onobject detection and time series migrationCONG Shuai¹,YANG Lei²,HUA Zhenghao³,YANG Xiaohui²(1. Industrial and Commercial College, Hebei University, Baoding 071000, China;2. Office of Educational Administration, Hebei University, Baoding 071002, China;3. School of Cyber Security and Computer, Hebei University, Baoding 071000, China )Abstract: In order to solve the problem of efficient supervision and management and data analysis of classroom persons in the intelligent reconstruction of education, this paper combines the good characteristics of single-stage object detection algorithm and the good special extraction ability of convolutional neural networks(CNN), and proposes an improved YOLO object detection algorithm based on attention network and time series migration. Firstly, this paper performs frame-by-frame extraction and non-distortion image deflation on the source video stream, and performs image super-resolution processing- 收稿日期:2022-09-13 基金项目:国家自然科学基金面上项目(62172139);中国高校产学研创新基金资助项目(2021BCF03002) 第一作者:丛帅(1980—),男,河北保定人,河北大学工商学院讲师,主要从事大数据技术、图像处理方面研究.E-mail:congshuai@hbu.cn 通信作者:杨磊(1986—),男,河北保定人,河北大学助理研究员,主要从事教育技术、智慧教学方面研究.E-mail:403064340@qq.com第4期丛帅等:基于目标检测和迁移时间序列的教室人员检测by Generative Adversarial Network(GAN). Secondly, multi-scale sampling and preliminary object detection are performed for each frame, and then Non Maximum Suppression(NMS)is performed to remove the individuals with low confidence based on the candidate results obtained at different scales. Then, the candidate results are fused, and the overlap is calculated using intersection over union(IoU)to update the data and remove overlapping or too close localization locations. Finally, the detection results of the current frame and those of the previous time interval are used as a time series for statistal data migration to obtain the final detection results. The experimental results show that this method not only effectively improves the accuracy of classroom personnel target detection, but also enables real-time detection.

Key words: object detection, multi-scale image features, super-resolution, attention-network, time series migration

中图分类号:

TP391.41

丛帅,杨磊,华征豪,杨晓晖. 基于目标检测和迁移时间序列的教室人员检测[J]. 河北大学学报(自然科学版), 2023, 43(4): 432-441.

CONG Shuai,YANG Lei,HUA Zhenghao,YANG Xiaohui. Classroom personnel detection based on object detection and time series migration[J]. Journal of Hebei University(Natural Science Edition), 2023, 43(4): 432-441.

参考文献

[1] 蔡宝来.教育信息化2.0时代的课堂变革:实质、理念及场景[J].海南师范大学学报(社会科学版), 2019, 32(4):82-88.DOI:10.16061/j.cnki.cn46-1076/c.2019.04.011.
[2] 童莹,沈越泓,魏以民.基于旋转主方向梯度直方图特征的判别稀疏图映射算法[J].物理学报,2019,68(19):95-110. DOI: 10.7498/aps.68.20190224.
[3] 洪志恒,陈明,秦玉芳,等.基于可变形部件模型的渔船安全监控系统[J].计算机应用与软件, 2018, 35(2):188-193. DOI: 10.3969/j.issn.1000-386x.2018.02.035.
[4] 程耀瑜,丰婧,李树军,等.一种基于Haar和肤色分割算法的人脸检测[J].兵器装备工程学报, 2021, 42(1):254-258. DOI: 10.11809/bqzbgcxb2021.01.046.
[5] HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[J]. IEEE Trans Pattern Anal Mach Intell, 2020, 42(2):386-397. DOI: 10.1109/TPAMI.2018.2844175.
[6] 薛超,于宏志,王景彬.基于卷积神经网络的级联人脸检测[J].中国安防,2017(11):88-91. DOI: 10.3969/j.issn.1673-7873.2017.11.021.
[7] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C] //2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016, Las Vegas, NV, USA. IEEE, 2016: 779-788. DOI: 10.1109/CVPR.2016.91.
[8] LIN T Y, DOLLáR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA, IEEE, 2017: 936-944. DOI: 10.1109/CVPR.2017.106.
[9] LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA. IEEE, 2017: 105-114. DOI: 10.1109/CVPR.2017.19.
[10] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv preprint arXiv:1409.1556, 2014. DOI: 10.48550/arXiv.1409.1556.
[11] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Salt Lake City, UT, USA. IEEE, 2018: 7132-7141. DOI: 10.1109/CVPR.2018.00745.
[12] LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Salt Lake City, UT, USA, IEEE, 2018: 8759-8768. DOI: 10.1109/CVPR.2018.00913.
[13] HE M, LUO H B, HUI B, et al. Fast online multi-pedestrian tracking via integrating motion model and deep appearance model[J]. IEEE Access, 2019, 7: 89475-89486. DOI: 10.1109/ACCESS.2019.2926416.
[14] HOSANG J, BENENSON R, SCHIELE B. Learning non-maximum suppression[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA, IEEE, 2017: 6469-6477. DOI: 10.1109/CVPR.2017.685.
[15] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C] //2017 IEEE International Conference on Computer Vision(ICCV), Venice, Italy, IEEE, 2017: 2999-3007. DOI: 10.1109/ICCV.2017.324.
[16] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[M] //Computer Vision - ECCV 2016, Cham: Springer International Publishing, 2016: 21-37. DOI: 10.1007/978-3-319-46448-0_2.
[17] WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C] //2017 IEEE International Conference on Image Processing(ICIP), Beijing, China, IEEE, 2018: 3645-3649. DOI: 10.1109/ICIP.2017.8296962. (