One-versus-one weighted twin support vector machine with pinball loss function

doi:10.3969/j.issn.1000-1565.2020.06.013

Abstract

Abstract: Twin support vector machine obtains a pair of non-parallel hyperplanes by solving two smaller quadratic programming problems, which improves the performance of the classifier in terms of time and accuracy. Because this method uses the Hinge loss function, the twin support vector function is more sensitive to noise and the resampling is unstable. Therefore, in view of the multi-classification problem, the pinball loss function and weights of samples are introduced into the twin support vector machine, and the binary classifiers obtained are combined by a one-versus-one method,and a one-versus-one weighted twin support vector machine based on pinball loss is proposed. It solves the problems of the sensitivity of twin support vector machines to noise and the instability of resampling. In addition, for the different effects of samples, some methods for calculating the weights of samples are given. In the experiment, standard data sets and artificial synthetic data sets are selected to verify the proposed algorithm Pin-OVO-TWSVM and compared the performance with OVO-TWSVM, OVA-TWSVM and Pin-OVO-TWSVM to show the effectiveness of the proposed method.

Key words: multi-classification, twin support vector machine, pinball loss, weight of sample

CLC Number:

TP391

LI Kai, LI Jie. One-versus-one weighted twin support vector machine with pinball loss function[J]. Journal of Hebei University(Natural Science Edition), 2020, 40(6): 647-656.

References

[1] VAPNIK V N. The nature of statistical learning theory[M]. New York: Springer New York, 1995. DOI:10.1007/978-1-4757-2440-0.
[2] JAYADEVA R K, KHEMCHANDANI R, CHANDRA S. Twin support vector machines for pattern classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(5): 905-910. DOI:10.1109/tpami.2007.1068.
[3] TOMAR D, AGARWAL S. Twin support vector machine: a review from 2007 to 2014[J]. Egyptian Informatics Journal, 2015, 16(1): 55-69. DOI:10.1016/j.eij.2014.12.003.
[4] SHAO Y H, ZHANG C H, WANG X B, et al. Improvements on twin support vector machines[J]. IEEE Transactions on Neural Networks, 2011, 22(6): 962-968. DOI:10.1109/tnn.2011.2130540.
[5] HUANG H J, WEI X X, ZHOU Y Q. Twin support vector machines: a survey[J]. Neurocomputing, 2018, 300: 34-43. DOI:10.1016/j.neucom.2018.01.093.
[6] DING S F, ZHANG N, ZHANG X K, et al. Twin support vector machine: theory, algorithm and applications[J]. Neural Computing and Applications, 2017, 28(11): 3119-3130. DOI:10.1007/s00521-016-2245-4.
[7] PENG X J. A ν-twin support vector machine(ν-TSVM)classifier and its geometric algorithms[J]. Information Sciences, 2010, 180(20): 3863-3875. DOI:10.1016/j.ins.2010.06.039.
[8] YU L A, ZHOU R T, TANG L, et al. A DBN-based resampling SVM ensemble learning paradigm for credit classification with imbalanced data[J]. Applied Soft Computing, 2018, 69: 192-202. DOI:10.1016/j.asoc.2018.04.049.
[9] KHEMCHANDANI R, SHARMA S. Robust least squares twin support vector machine for human activity recognition[J]. Applied Soft Computing, 2016, 47: 33-46. DOI:10.1016/j.asoc.2016.05.025.
[10] HUANG X, SHI L, SUYKENS J A. Support vector machine classifier with pinball loss[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 984-997. DOI:10.1109/tpami.2013.178.
[11] TANVEER M, SHARMA A, SUGANTHAN P N. General twin support vector machine with pinball loss function[J]. Information Sciences, 2019, 494: 311-327. DOI:10.1016/j.ins.2019.04.032.
[12] XU Y, YANG Z, PAN X. A novel twin support-vector machine with pinball loss[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(2): 359-370. DOI:10.1109/tnnls.2015.2513006.
[13] LIN C F, WANG S D. Fuzzy support vector machines with automatic membership setting[M] //Support Vector Machines: Theory and Applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005: 233-254. DOI:10.1007/10984697_11.
[14] LI K, MA H Y. A fuzzy twin support vector machine algorithm[J]. International Journal of Application or Innovation in Engineering & Management, 2013, 2(3):459-465.
[15] 丁胜锋.一种改进的双支持向量机[J].辽宁石油化工大学学报,2012,32(4):76-79,82. DOI:10.3969/j.issn.1672-6952.2012.04.020.
[16] 方佳艳,刘峤,吴德,等.基于模糊C-均值的相似性特征转换光滑支持向量机[J].电子学报,2018,46(11):2714-2724. DOI:10.3969/j.issn.0372-2112.2018.11.019.
[17] YANG X W, ZHANG G Q, LU J, et al. A kernel fuzzy c-means clustering-based fuzzy support vector machine algorithm for classification problems with outliers or noises[J]. IEEE Transactions on Fuzzy Systems, 2011, 19(1): 105-115. DOI:10.1109/tfuzz.2010.2087382.
[18] CHU M X, LIU X P, GONG R F, et al. Multi-class classification method using twin support vector machines with multi-information for steel surface defects[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 176: 108-118. DOI:10.1016/j.chemolab.2018.03.014.
[19] HSU C W, LIN C J. A comparison of methods for multiclass support vector machines[J]. IEEE Transactions on Neural Networks, 2002, 13(2): 415-425. DOI:10.1109/72.991427.
[20] 丁世飞,张健,张谢锴,等.多分类孪生支持向量机研究进展[J].软件学报,2018,29(1):89-108. DOI:10.13328/j.cnki.jos.005319.
[21] XU Y T, GUO R, WANG L S. A twin multi-class classification support vector machine[J]. Cognitive Computation, 2013, 5(4): 580-588. DOI:10.1007/s12559-012-9179-7.
[22] YANG Z X, SHAO Y H, ZHANG X S. Multiple birth support vector machine for multi-class classification[J]. Computing and Applications, 2013, 22(1): 153-161. DOI:10.1007/s00521-012-1108-x.
[23] BLACK C, MERS C J. UCI Repository for machine learning databases[EB/OL]. Irvine, CA: University of California, School of Information and Computer Science.(1998-01-12)[2018-11-20].http://archive.ics.uci.edu/ml.