Malaria is a serious and fatal infectious disease and early detection of the patients infection severity can effectively curb the outbreak of this infectious disease. Deep learning has been verified to have excellent capability in image classification and disease diagnosis in many challenging tasks, such as cell detection and histological image classification. There exist many deep learning studies on malaria parasite recognition with successful applications, but they mainly focus on binary classification of single ring stage and red blood cells. The most important disadvantages of them are ignoring other stages of malaria parasites, including Trophozoite, Gametocytes and Schizont. In this paper, we are the first to study the multi-stage malaria parasite recognition problem, and propose a novel Neighbor Sample Joint Learning(NSJL)for this challenging task. Specifically, NSJL consists of CNN(Convolutional Neural Network)feature learning, neighbour correlation mining and graph representation modules. The method firstly extracts CNN representations from each parasite image and then establishes the neighbour correlations among CNN features by K-Nearest Neighbour(K-NN)graph building algorithms, with operating Graph Convolutional Network(GCN)on CNN features and their correlations. To evaluate the performance of our- DOI:10.3969/j.issn.1000-1565.2022.02.015基于近邻样本联合学习模型的疟疾识别算法哈艳^1,2,孟翔杰³,田俊峰^2,4(1.河北大学 管理学院,河北 保定 071002;2.河北省高可信信息系统重点实验室,河北 保定 071002;3.河北大学 数学与信息科学学院,河北 保定 071002;4.河北大学 网络空间安全与计算机学院,河北 保定 071002)摘 要:疟疾早期诊断可以有效防止疾病暴发.深度学习在细胞形态和组织图像检测等任务中具有出色能力.已有许多基于深度学习的疟疾研究,但它们主要用于环状体和红细胞二分类.本文首次研究疟疾多阶段识别,并提出近邻样本联合学习(neighbor sample joint learning,NSJL)模型.NSJL包括卷积神经网络(convolutional neural network,CNN)特征学习、领域相关性挖掘和图特征嵌入.它提取CNN特征,并将其与K近邻(K-nearset neighbor,K-NN)建立的邻域图传入图卷积网络(graph convolutional network,GCN).为评估NSJL,将其与先进方法比较,结果表明NSJL模型可达92.50%准确率,92.84%精确度,92.50%召回率和92.52%F1分数,至少高于其他方法7%的准确率表明其优秀疟疾识别能力.关键词:疟原虫识别;图卷积网络;多阶段分类;样本联系;深度学习.中图分类号:TP391 文献标志码:A 文章编号:1000-1565(2022)02-0208-09Neighbor sample joint learning for malaria parasite recognitionHA Yan^1,2,MENG Xiangjie³,TIAN Junfeng^2,4(1. School of Management, Hebei University, Baoding 071002,China;2. Key Laboratory on High Trusted Information System in Hebei Province, Baoding 071002,China; 3. College of Mathematics and Information Science, Hebei University, Baoding 071002,China; 4. School of Cyber Security and Computer, Hebei University, Baoding 071002,China)Abstract: Malaria is a serious and fatal infectious disease and early detection of the patients infection severity can effectively curb the outbreak of this infectious disease. Deep learning has been verified to have excellent capability in image classification and disease diagnosis in many challenging tasks, such as cell detection and histological image classification. There exist many deep learning studies on malaria parasite recognition with successful applications, but they mainly focus on binary classification of single ring stage and red blood cells. The most important disadvantages of them are ignoring other stages of malaria parasites, including Trophozoite, Gametocytes and Schizont. In this paper, we are the first to study the multi-stage malaria parasite recognition problem, and propose a novel Neighbor Sample Joint Learning(NSJL)for this challenging task. Specifically, NSJL consists of CNN(Convolutional Neural Network)feature learning, neighbour correlation mining and graph representation modules. The method firstly extracts CNN representations from each parasite image and then establishes the neighbour correlations among CNN features by K-Nearest Neighbour(K-NN)graph building algorithms, with operating Graph Convolutional Network(GCN)on CNN features and their correlations. To evaluate the performance of our- 收稿日期:2021-05-26 基金项目:国家自然科学基金资助项目(61802106);河北省自然科学基金资助项目(F2021201049) 第一作者:哈艳(1974—),女,回族,河北肃宁人,河北大学教授,河北大学在读博士,主要从事深度学习、可信计算方向研究.E-mail:hayanhbu@163.com第2期哈艳等:基于近邻样本联合学习模型的疟疾识别算法NSJL model, we compare it with several advanced existing methods, and our model can achieve a high accuracy of 92.50%, precision of 92.84%, recall of 92.50% and F1-score of 92.52%. The comparison with existing advanced methods verifies the NSJL model has excellent capability for recognizing multi-stage malaria parasite, which is higher than the compared methods by at least 7% in accuracy.