基于pinball损失的一对一加权孪生支持向量机

doi:10.3969/j.issn.1000-1565.2020.06.013

摘要/Abstract

摘要： 孪生支持向量机通过求解2个较小二次规划问题得到一对非平行超平面,从时间和准确率方面提高了分类器的性能.由于此方法使用Hinge损失函数,造成孪生支持向量机对噪声较为敏感以及重采样的不稳定.为此,针对多分类问题,将pinball损失函数与样本权重引入到孪生支持向量机中,采用一对一方法组合二分类器,提出了基于pinball损失的一对一加权孪生支持向量机,较好地解决了孪生支持向量机对噪声的敏感性以及重采样的不稳定性.另外,对于样本的不同影响,给出了多种求取样本权重的方法.实验中选取标准数据集和人工合成数据集对提出的算法进行了验证,并与一对一孪生支持向量机(OVO-TWSVM)、一对多孪生支持向量机(OVA-TWSVM)以及基于pinball损失的一对一加权孪生支持向量机(Pin-OVO-TWSVM)进行了比较,表明了提出方法的有效性.

关键词: 多分类, 孪生支持向量机, pinball损失, 样本权重

Abstract: Twin support vector machine obtains a pair of non-parallel hyperplanes by solving two smaller quadratic programming problems, which improves the performance of the classifier in terms of time and accuracy. Because this method uses the Hinge loss function, the twin support vector function is more sensitive to noise and the resampling is unstable. Therefore, in view of the multi-classification problem, the pinball loss function and weights of samples are introduced into the twin support vector machine, and the binary classifiers obtained are combined by a one-versus-one method,and a one-versus-one weighted twin support vector machine based on pinball loss is proposed. It solves the problems of the sensitivity of twin support vector machines to noise and the instability of resampling. In addition, for the different effects of samples, some methods for calculating the weights of samples are given. In the experiment, standard data sets and artificial synthetic data sets are selected to verify the proposed algorithm Pin-OVO-TWSVM and compared the performance with OVO-TWSVM, OVA-TWSVM and Pin-OVO-TWSVM to show the effectiveness of the proposed method.

Key words: multi-classification, twin support vector machine, pinball loss, weight of sample

中图分类号:

TP391

李凯,李洁. 基于pinball损失的一对一加权孪生支持向量机[J]. 河北大学学报(自然科学版), 2020, 40(6): 647-656.

LI Kai, LI Jie. One-versus-one weighted twin support vector machine with pinball loss function[J]. Journal of Hebei University(Natural Science Edition), 2020, 40(6): 647-656.

参考文献

[1] VAPNIK V N. The nature of statistical learning theory[M]. New York: Springer New York, 1995. DOI:10.1007/978-1-4757-2440-0.
[2] JAYADEVA R K, KHEMCHANDANI R, CHANDRA S. Twin support vector machines for pattern classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(5): 905-910. DOI:10.1109/tpami.2007.1068.
[3] TOMAR D, AGARWAL S. Twin support vector machine: a review from 2007 to 2014[J]. Egyptian Informatics Journal, 2015, 16(1): 55-69. DOI:10.1016/j.eij.2014.12.003.
[4] SHAO Y H, ZHANG C H, WANG X B, et al. Improvements on twin support vector machines[J]. IEEE Transactions on Neural Networks, 2011, 22(6): 962-968. DOI:10.1109/tnn.2011.2130540.
[5] HUANG H J, WEI X X, ZHOU Y Q. Twin support vector machines: a survey[J]. Neurocomputing, 2018, 300: 34-43. DOI:10.1016/j.neucom.2018.01.093.
[6] DING S F, ZHANG N, ZHANG X K, et al. Twin support vector machine: theory, algorithm and applications[J]. Neural Computing and Applications, 2017, 28(11): 3119-3130. DOI:10.1007/s00521-016-2245-4.
[7] PENG X J. A ν-twin support vector machine(ν-TSVM)classifier and its geometric algorithms[J]. Information Sciences, 2010, 180(20): 3863-3875. DOI:10.1016/j.ins.2010.06.039.
[8] YU L A, ZHOU R T, TANG L, et al. A DBN-based resampling SVM ensemble learning paradigm for credit classification with imbalanced data[J]. Applied Soft Computing, 2018, 69: 192-202. DOI:10.1016/j.asoc.2018.04.049.
[9] KHEMCHANDANI R, SHARMA S. Robust least squares twin support vector machine for human activity recognition[J]. Applied Soft Computing, 2016, 47: 33-46. DOI:10.1016/j.asoc.2016.05.025.
[10] HUANG X, SHI L, SUYKENS J A. Support vector machine classifier with pinball loss[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 984-997. DOI:10.1109/tpami.2013.178.
[11] TANVEER M, SHARMA A, SUGANTHAN P N. General twin support vector machine with pinball loss function[J]. Information Sciences, 2019, 494: 311-327. DOI:10.1016/j.ins.2019.04.032.
[12] XU Y, YANG Z, PAN X. A novel twin support-vector machine with pinball loss[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(2): 359-370. DOI:10.1109/tnnls.2015.2513006.
[13] LIN C F, WANG S D. Fuzzy support vector machines with automatic membership setting[M] //Support Vector Machines: Theory and Applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005: 233-254. DOI:10.1007/10984697_11.
[14] LI K, MA H Y. A fuzzy twin support vector machine algorithm[J]. International Journal of Application or Innovation in Engineering & Management, 2013, 2(3):459-465.
[15] 丁胜锋.一种改进的双支持向量机[J].辽宁石油化工大学学报,2012,32(4):76-79,82. DOI:10.3969/j.issn.1672-6952.2012.04.020.
[16] 方佳艳,刘峤,吴德,等.基于模糊C-均值的相似性特征转换光滑支持向量机[J].电子学报,2018,46(11):2714-2724. DOI:10.3969/j.issn.0372-2112.2018.11.019.
[17] YANG X W, ZHANG G Q, LU J, et al. A kernel fuzzy c-means clustering-based fuzzy support vector machine algorithm for classification problems with outliers or noises[J]. IEEE Transactions on Fuzzy Systems, 2011, 19(1): 105-115. DOI:10.1109/tfuzz.2010.2087382.
[18] CHU M X, LIU X P, GONG R F, et al. Multi-class classification method using twin support vector machines with multi-information for steel surface defects[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 176: 108-118. DOI:10.1016/j.chemolab.2018.03.014.
[19] HSU C W, LIN C J. A comparison of methods for multiclass support vector machines[J]. IEEE Transactions on Neural Networks, 2002, 13(2): 415-425. DOI:10.1109/72.991427.
[20] 丁世飞,张健,张谢锴,等.多分类孪生支持向量机研究进展[J].软件学报,2018,29(1):89-108. DOI:10.13328/j.cnki.jos.005319.
[21] XU Y T, GUO R, WANG L S. A twin multi-class classification support vector machine[J]. Cognitive Computation, 2013, 5(4): 580-588. DOI:10.1007/s12559-012-9179-7.
[22] YANG Z X, SHAO Y H, ZHANG X S. Multiple birth support vector machine for multi-class classification[J]. Computing and Applications, 2013, 22(1): 153-161. DOI:10.1007/s00521-012-1108-x.
[23] BLACK C, MERS C J. UCI Repository for machine learning databases[EB/OL]. Irvine, CA: University of California, School of Information and Computer Science.(1998-01-12)[2018-11-20].http://archive.ics.uci.edu/ml.