Robust fuzzy least squares twin parametric-margin support vector machine algorithm

doi:10.3969/j.issn.1000-1565.2024.06.011

Abstract

Abstract: As least squares twin parameter-margin support vector machines(LSTPMSVM)are sensitive to noise and susceptible to outliers in the process of classification, a robust fuzzy least squares twin parameter-margin support vector machine(RFLSTPMSVM)algorithm is proposed. The algorithm uses 2-norm of the slack variables to make the optimization problem strongly convex, and then assigns appropriate weights to each data sample based on the fuzzy affiliation values, which reduces the influence of outliers effectively. At the same time, the algorithm introduces K-nearest neighbour weighting into the objective function, considering the local information of samples and improving the accuracy of the model. In addition, the algorithm is optimised by solving a simple system of linear equations, rather than solving quadratic programming problems, giving the model a faster computational speed. The proposed algorithm is assessed and compared with TWSVM, LSTSVM, LSTPMSVM and ULSTPMSVM on some UCI datasets.- DOI:10.3969/j.issn.1000-1565.2024.06.011鲁棒的模糊最小二乘双参数间隔支持向量机算法杨贵燕¹,黄成泉²,罗森艳¹,蔡江海¹,王顺霞¹,周丽华¹(1.贵州民族大学数据科学与信息工程学院,贵州贵阳 550025;2.贵州民族大学工程技术人才实践训练中心,贵州贵阳 550025)摘要:针对最小二乘双参数间隔支持向量机(LSTPMSVM)对噪声敏感且在分类过程中易受异常值影响的问题,提出了一种鲁棒的模糊最小二乘双参数间隔支持向量机算法(RFLSTPMSVM).该算法利用松弛变量的2范数使得优化问题具有强凸性,再根据隶属度为每个样本分配相应的权重,有效降低异常值带来的影响.同时,在目标函数中引入K-近邻加权,考虑样本之间的局部信息,提高模型的分类准确率.此外,通过求解简单的线性方程组来优化该算法,而不是求解二次规划问题,使模型具有较快的计算速度.在UCI(university of California irvine)数据集上对该算法进行性能评估,并与TWSVM、LSTSVM、LSTPMSVM和ULSTPMSVM 4种算法进行比较.数值实验结果表明,该算法具有更好的泛化性能.关键词:双参数间隔支持向量机;孪生支持向量机;模糊隶属度;K-近邻中图分类号:TP181 文献标志码:A 文章编号:1000-1565(2024)06-0653-13Robust fuzzy least squares twin parametric-margin support vector machine algorithmYANG Guiyan¹, HUANG Chengquan², LUO Senyan¹, CAI Jianghai¹, WANG Shunxia¹, ZHOU Lihua¹(1.School of Data Science and Information Engineering, Guizhou Minzu University,Guiyang 550025,China; 2. Engineering Training Center, Guizhou Minzu University, Guiyang 550025, China)Abstract: As least squares twin parameter-margin support vector machines(LSTPMSVM)are sensitive to noise and susceptible to outliers in the process of classification, a robust fuzzy least squares twin parameter-margin support vector machine(RFLSTPMSVM)algorithm is proposed. The algorithm uses 2-norm of the slack variables to make the optimization problem strongly convex, and then assigns appropriate weights to each data sample based on the fuzzy affiliation values, which reduces the influence of outliers effectively. At the same time, the algorithm introduces K-nearest neighbour weighting into the objective function, considering the local information of samples and improving the accuracy of the model. In addition, the algorithm is optimised by solving a simple system of linear equations, rather than solving quadratic programming problems, giving the model a faster computational speed. The proposed algorithm is assessed and compared with TWSVM, LSTSVM, LSTPMSVM and ULSTPMSVM on some UCI datasets.- 收稿日期:2024-06-06;修回日期:2024-07-10 基金项目:国家自然科学基金资助项目(62062024);贵州省省级科技计划项目(黔科合基础-ZK［2021］一般342);贵州省教育厅自然科学研究项目(黔教技［2022］015);贵州省模式识别与智能系统重点实验室2022年度开放课题(GZMUKL［2022］KF03) 第一作者:杨贵燕(1997—),女,贵州民族大学在读硕士研究生,主要从事机器学习、模式识别等研究. E-mail:2393350042@qq.com 通信作者:黄成泉(1976—),男,贵州民族大学教授,博士,主要从事机器学习、模式识别、图像处理等研究. E-mail:hcq@gzmu.edu.cn. 第6期杨贵燕等:鲁棒的模糊最小二乘双参数间隔支持向量机算法河北大学学报(自然科学版) 第44卷The numerical experiments results show that the proposed algorithm has better generalization performance.

Key words: twin parametric-margin support vector machine, twin support vector machine, fuzzy membership, K-nearest neighbor(K-NN)

CLC Number:

TP181

YANG Guiyan, HUANG Chengquan, LUO Senyan, CAI Jianghai, WANG Shunxia, ZHOU Lihua. Robust fuzzy least squares twin parametric-margin support vector machine algorithm[J]. Journal of Hebei University(Natural Science Edition), 2024, 44(6): 653-665.

References

[1] CORTES C, VAPNIK V. Support-vector networks[J]. Mach Learn, 1995, 20(3): 273-297. DOI: 10.1007/ bf00994018.
[2] SUBUDHIRAY S, PALO H K, DAS N. Effective recognition of facial emotions using dual transfer learned feature vectors and support vector machine[J]. Int J Inf Technol, 2023, 15(1): 301-313. DOI:10.1007/s41870-022-01093-7.
[3] PANG J X, PU X K, LI C G. A hybrid algorithm incorporating vector quantization and one-class support vector machine for industrial anomaly detection[J]. IEEE Trans Ind Inform, 2022, 18(12): 8786-8796. DOI:10.1109/TII.2022.3145834.
[4] YI L, XIE G J, LI Z H, et al. Automatic depression diagnosis through hybrid EEG and near-infrared spectroscopy features using support vector machine[J]. Front Neurosci, 2023, 17: 1205931. DOI:10.3389/fnins.2023.1205931.
[5] MANGASARIAN O L, MUSICANT D R. Lagrangian support vector machines[J]. J Mach Learn Research, 2001, 1(Mar): 161-177. DOI:10.1162/15324430152748218.
[6] SCHÖLKOPF B, SMOLA A J, WILLIAMSON R C, et al. New support vector algorithms[J]. Neural Comput, 2000, 12(5): 1207-1245. DOI:10.1162/089976600300015565.
[7] LIN C F, WANG S D. Fuzzy support vector machines[J]. IEEE Trans Neural Netw, 2002, 13(2): 464-471. DOI:10.1109/72.991432.
[8] JAYADEVA, KHEMCHANDANI R, CHANDRA S. Twin support vector machines for pattern classification[J]. IEEE Trans Pattern Anal Mach Intell, 2007, 29(5): 905-910. DOI:10.1109/ TPAMI. 2007.1068
[9] ARUN KUMAR M, GOPAL M. Least squares twin support vector machines for pattern classification[J]. Expert Syst Appl, 2009, 36(4): 7535-7543. DOI:10.1016/j.eswa.2008.09.066.
[10] PENG X J. TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition[J]. Pattern Recognit, 2011, 44(10/11): 2678-2692. DOI:10.1016/j.patcog.2011.03.031.
[11] SHAO Y H, WANG Z, CHEN W J, et al. Least squares twin parametric-margin support vector machine for classification[J]. Appl Intell, 2013, 39(3): 451-464. DOI:10.1007/s10489-013-0423-y.
[12] WANG H R, XU Y T, ZHOU Z J. Twin-parametric margin support vector machine with truncated pinball loss[J]. Neural Comput Appl, 2021, 33(8): 3781-3798. DOI:10.1007/s00521-020-05225-7.
[13] RASTOGI NéE KHEMCHANDANI R, SAIGAL P, CHANDRA S. Angle-based twin parametric-margin support vector machine for pattern classification[J]. Knowl Based Syst, 2018, 139: 64-77. DOI:10.1016/j.knosys. 2017.10.008.
[14] GUPTA D, BORAH P, PRASAD M. A fuzzy based Lagrangian twin parametric-margin support vector machine(FLTPMSVM)[C] //2017 IEEE Symposium Series on Computational Intelligence(SSCI), Honolulu, HI, USA. IEEE, 2017: 1-7. DOI:10.1109/SSCI.2017.8280964.
[15] RICHHARIYA B, TANVEER M. Universum least squares twin parametric-margin support vector machine[C] //2020 International Joint Conference on Neural Networks(IJCNN), Glasgow, UK, IEEE, 2020: 1-8. DOI: 10.1109/IJCNN48605.2020.9206865.
[16] TANVEER M, SHUBHAM K, ALDHAIFALLAH M, et al. An efficient regularized K-nearest neighbor based weighted twin support vector regression[J]. Knowl Based Syst, 2016, 94: 70-87. DOI: 10.1016/j.knosys.2015.11.011.
[17] RICHHARIYA B, TANVEER M, FOR THE ALZHEIMER’S DISEASE NEUROIMAGING INITIATIVE. A fuzzy universum least squares twin support vector machine(FULSTSVM)[J]. Neural Comput Appl, 2022, 34(14): 11411-11422. DOI:10.1007/s00521-021-05721-4.
[18] DUA D, TANISKIDOU E K. UCI machine learning repository, 2017[DB/OL].[2022-10-16]. http://archive. ics. uci. edu/ml/.
[19] 刘小生, 章治邦.基于改进网格搜索法的SVM参数优化[J].江西理工大学学报, 2019, 40(1): 5-9. DOI: 10.13265/j.cnki.jxlgdxxb.2019.01.002.
[20] BENAVOLI A, CORANI G, DEMSAR J, et al. Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis[J]. J Mach Learn Research, 2017, 18(77): 1-36. DOI:10. 48550/arXiv.1606.04316. (