基于向量空间的文档相似度算法假设特征元素间关系为正交,当2篇文档采用了具有相近语义的不同术语描述时,该方法不能准确反映二者的相似性.针对这种情况,文章利用词语的同义关系,在给出术语与术语组相似度、术语组和术语组间相似度的概念及算法的基础上,给出一种基于词语相似关系的文档相似度计算方法.实验采用科技文献类文档和新闻报道类文档作为测试集合,比较新方法和向量空间算法的分类性能,结果显示新方法可提高文档分类的准确性.
Because vector space model(VSM)assumes that terms in different documents is orthogonal,when different documents are described by different terms,VSM can’t accurately reflect the similarity between them.For this problem,based on giving definition and computing method of similarity between two terms set,this paper gives a quantification method to calculate similarity between two documents.Our experiments adopt science and technology literature documents and news stories to test the classification accuracy of VSM and the new method,results indicate that the new method can improve classification accuracy.
[1] 孙润志.基于语义理解的文本相似度计算研究与实现 [D].沈阳:中国科学院沈阳计算技术研究所,2015. SUN R Z.Research and implementation of text similarity computing based on semantic understanding [D].Shenyang:Shenyang Institute of Computing Technology,Chinese Academy of Sciences,2015.
[2] 杨长春,徐小松,叶施仁,等.基于文本相似度的微博网络水军发现算法[J].微电子学与计算机,2014,31(3):82-85. YANG C C,XU X S,YE S R,et al.A method to find water armies in weibo based on text similarity[J].MICROELECTRONICS & COMPUTER,2014,31(3):82-85.
[3] 谭静.基于向量空间模型的文本相似度算法研究 [D].成都:西南石油大学,2015.
[4] RICARDO BAEZA-YATES,BARTHIER RIBEIRO-NETO.Mordern informtion retrieval[M].北京:机械工业出版社,2004:24-38.
[5] 赵宁宁,梁意文.综合结构和内容的XML文档相似度计算方法 [J].微电子学与计算机,2016,33(4):69-72.DOI:10.19304/j.cnki.issn1000-7180.2016.04.015. ZHAO N N, LIANG Y W. Combining structure and content similaritiesmeasure for XML document[J].Microelectronics & Computer,2016,33(4):69-72.DOI:10.19304/j.cnki.issn1000-7180.2016.04.015.
[6] 万昊,谭宗颖,鲁晶晶,等.2001~2014年引文分析领域发展演化综述 [J].图书情报工作,2015,59(6):120-136.DOI:10.13266/j.issn.0252-3116.2015.06.018.DOI:10. 13266 / j. issn. 0252 - 3116. 2015. 06. 018.
[7] CAMPOS L M de,FERNáNDEZ-LUNA J M,HUET J F.Clustering terms in the Bayesian network retrieval model:a new approach with two term-layers[J].Applied Soft Computing,2004,4(2):149-158.DOI:http://dx.doi.org/10.1016/j.asoc.2003.11.003.
[8] 牛萍,黄德根.TF-IDF与规则相结合的中文关键词自动抽取研究 [J].小型微型计算机系统,2016,37(4):711-715. NIU P,HUANG D G.TF-IDF and rules based automatic extraction of chinese keywords [J].Journal of Chinese Computer Systems,2016,37(4):711-715.
[9] 张为泰.基于词向量模型特征空间优化的同义词扩展研究与应用 [D].北京:北京邮电大学,2015.
[10] 刘怀亮,杜坤,秦春秀.基于知网语义相似度的中文文本分类研究 [J].现代图书情报技术,2015,2:39-45.
[11] 徐建民.基于术语关系的贝叶斯网络信息检索模型扩展研究[D].天津:天津大学,2007. XU J M.Research of Using term-relationships to extend Bayesian network retrieval models[D].Tianjin:Tianjin University,2007.
[12] 陈宏朝,李飞,朱新华,等.基于路径与深度的同义词词林词语相似度计算 [J].中文信息学报, 2016,30(5):80-88. CHEN H C,LI F,ZHU X H,et al.A path and depth-based approach to word semantic similarity calcalation in CiLin[J].Journal of Chinese Information Processing,2016,30(5):80-88.
