现有全文检索技术多是以文本信息为处理对象,对于以数学表达式为主要成分的科技文档检索还处在探索阶段.为了使用户可以方便地以数学公式作为查询语言对科技文档进行检索,提出了一种基于数学表达式特征的科技文档检索模型.首先通过将公式解析为二叉树得到数学表达式的子式信息,利用数学表达式及子式构造检索特征向量;在索引阶段,利用所提取的文档特征向量构建分层结构的索引表;在匹配阶段,对文档向量采用tf-idf进行加权操作,利用余弦相似度对检索向量和文档向量进行相似度计算,得到一个有序的文档检索结果.实验选取了来自不同领域的期刊、学术网站以及公共数据集的5 017篇科技文档,其中包含了96 362条数学公式,平均检索时间为0.428 s,表明该模型达到了实现较高效率科技文档检索的目标.
The existing full-text retrieval techniques are mostly targeting the text information.While the retrieval of the scientific documents with mathematical expressions as the main components is still in the exploration stage.In order to make the users can easily use the mathematical formula as the query language to retrieve the scientific and technical documents,a new scientific document retrieval model based on mathematical expression features was proposed.Firstly,the formulas were resolved into the subformulas forming the binary trees,which are used to generate the retrieval feature vectors.In the index phase,the index table with the hierarchical structure was constructed using the extracted document feature vectors.In the retrieval phase,the document vectors were weighted by tf-idf.The similarity between the retrieval vector and the document vector was calculated by using the cosine similarity,and an ordered document retrieval result was obtained.The experiment data was selected from different journals,academic website and public data set of 5 017 science and technology documents which contain 96 362 mathematical formulas.The average retrieval time was 0.428 s,which indicates that the proposed model achieved- DOI:10.3969/j.issn.1000-1565.2017.06.013基于数学表达式特征的科技文档检索模型田学东,崔晓娟(河北大学 计算机科学与技术学院,河北 保定 071002)摘 要:现有全文检索技术多是以文本信息为处理对象,对于以数学表达式为主要成分的科技文档检索还处在探索阶段.为了使用户可以方便地以数学公式作为查询语言对科技文档进行检索,提出了一种基于数学表达式特征的科技文档检索模型.首先通过将公式解析为二叉树得到数学表达式的子式信息,利用数学表达式及子式构造检索特征向量;在索引阶段,利用所提取的文档特征向量构建分层结构的索引表;在匹配阶段,对文档向量采用tf-idf进行加权操作,利用余弦相似度对检索向量和文档向量进行相似度计算,得到一个有序的文档检索结果.实验选取了来自不同领域的期刊、学术网站以及公共数据集的5 017篇科技文档,其中包含了96 362条数学公式,平均检索时间为0.428 s,表明该模型达到了实现较高效率科技文档检索的目标.关键词:科技文档;数学表达式;检索;索引;匹配;二叉树;特征中图分类号:TP391 文献标志码:A 文章编号:1000-1565(2017)06-0652-10A retrieval model of scientific documents based onmathematical expression featuresTIAN Xuedong,CUI Xiaojuan(College of Computer Science and Technology,Hebei University,Baoding 071002,China)Abstract: The existing full-text retrieval techniques are mostly targeting the text information.While the retrieval of the scientific documents with mathematical expressions as the main components is still in the exploration stage.In order to make the users can easily use the mathematical formula as the query language to retrieve the scientific and technical documents,a new scientific document retrieval model based on mathematical expression features was proposed.Firstly,the formulas were resolved into the subformulas forming the binary trees,which are used to generate the retrieval feature vectors.In the index phase,the index table with the hierarchical structure was constructed using the extracted document feature vectors.In the retrieval phase,the document vectors were weighted by tf-idf.The similarity between the retrieval vector and the document vector was calculated by using the cosine similarity,and an ordered document retrieval result was obtained.The experiment data was selected from different journals,academic website and public data set of 5 017 science and technology documents which contain 96 362 mathematical formulas.The average retrieval time was 0.428 s,which indicates that the proposed model achieved- 收稿日期:2017-09-18 基金项目:国家自然科学基金资助项目(61375075);河北省教育厅河北省高等学校科学技术研究重点项目(ZD2017208) 第一作者:田学东(1963—),男,天津人,河北大学教授,博士,主要从事模式识别、信息检索的研究. E-mail:xuedong_tian@126.com第6期田学东等:基于数学表达式特征的科技文档检索模型the goal of realizing mathematical expression retrieval with high efficiency.
[1] MINER R,MUNAVALLI R.An approach to mathematical search through query formulation and data normalization[M] //Towards Mechanized Mathematical Assistants,Springer Berlin Heidelberg,2007: 342-355.DOI:10.1007/978-3-540-73086-6_27.
[2] LIBBRECHT P,MELIS E.Methods to access and retrieve mathematical content in activemath[M] // Mathematical Software-ICMS 2006,Springer Berlin Heidelberg,2006: 331-342.DOI: 10.1007/11832225_33.
[3] LIBBRECHT P,MELIS E.Semantic search in leactivemath[C] //Proceedings of the First WebALT Conference and Exhibition.Eindhoven,Holland,Jan.2006: 97-109.
[4] MIŠUTKA J,GALAMBOŠ L.System description: EgoMath2 as a tool for mathematical searching on wikipedia.org[M] //Intelligent Computer Mathematics,Springer Berlin Heidelberg,2011: 307-309.DOI:10.1007/978-3-642-22673-1_30.
[5] 景珂.网络数学搜索中的数学查询语言与索引的研究[D].兰州:兰州大学,2009. JING K.Research on math query language and index in web-based math search[D].Lanzhou: Lanzhou University of Technology,2011.
[6] GUO W,SU W,LI L,et al.MQL: A mathematical formula query language for mathematical search [C] // Proceedings of the IEEE Conference on Computational Science and Engineering,Dalian,China: IEEE Press,2011: 245-250.DOI:10.1109/CSE.2011.52.
[7] 刘志伟.数学搜索引擎研究[D].兰州: 兰州大学,2011. LIU Z W.The Research on mathematics search[D].Lanzhou: Lanzhou University of Technology,2011.
[8] SOJKA P,LíŠKA M.KA M.Indexing and searching mathematics in digital libraries[C] //Proceedings of International Conference on Intelligent Computer Mathematics,Berlin,Germany:Springer,2011: 228-243.DOI:10.1007/978-3-642-22673-1_16.
[9] YOUSSEF A.Methods of relevance ranking and hit-content generation in math search [M] // Towards Mechanized Mathematical Assistants,Springer Berlin Heidelberg,2007: 393-406.DOI:10.1007/978-3-540-73086-6_31.
[10] SHATNAWI M,YOUSSEF A.Equivalence detection using parse-tree normalization for math search [C] // Proceedings of the 2nd International Conference on Digital Information Management,Lyon,France:IEEE Press,2007,2: 643-648.DOI: 10.1109/ICDIM.2007.4444297.
[11] MILLER B,YOUSSEF A.Technical aspects of the digital library of mathematical functions[J].Annals of Mathematics and Artificial Intelligence,2003,38(1): 121-136.DOI: 10.1023/A:1022967814992.
[12] LíŠKA M,SOJKA P,RU。ZˇIC ˇKA M.Combining text and formula queries in math information retrieval: evaluation of query results merging strategies[C].Proceedings of the First International Workshop on Novel Web Search Interfaces and Systems.Melbourn,Australia,Oct.2015: 7-9.DOI: 10.1145/2810355.2810359.
[13] 孙净.基于Trie树的数学表达式运算结构检索[D].保定: 河北大学,2015. SUN J.Operation structural retrieval of mathematical expressions based on trie tree[D].Baoding: Hebei University,2015.
[14] 卢托.科技文档中数学公式的描述与检索[D].武汉: 华中科技大学,2007. LU T.The description and retrieval of math formulas in scientific documents[D].Wuhan: Huazhong University of Science and Technology,2007.
[15] KAMALI S,TOMPA F.Structural similarity search for mathematics retrieval [M].Intelligent Computer Mathematics,Springer Berlin Heidelberg,2013: 246-262.DOI:10.1007/978-3-642-39320-4_16.
[16] ZANIBBI R,YUAN B.Keyword and image-based retrieval for mathematical expressions[C] // Proceedings of the SPIE 7874 Conference on Document Recognition and Retrieval XVIII,California,USA: SPIE,2011,7874(6): 993-1004.DOI: 10.1117/12.873312.
[17] HU X,GAO L C,LIN X Y,et al.WikiMirs: A mathematical information retrieval system for wikipedia[C].Proceedings of the 13th ACM/IEEE-CS Joint Conference on Digital libraries.ACM,2013: 11-20.DOI:10.1145/2467696.2467699.
[18] LIN X Y,GAO L C,HU X,et al.A mathematics retrieval system for formulae in layout presentations [C] // Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval,Gold Coast,Australia: ACM,2014: 697-706.DOI:10.1145/2600428.2609611.
[19] SCHELLENBERG T,YUAN B,ZANIBBI R.Layout-based substitution tree indexing and retrieval for mathematical expressions [C] // Proceedings of the SPIE 7874 Conference on Document Recognition and Retrieval XIX,California,USA: SPIE,2012,8297(2): 263-271.DOI:10.1117/12.912502.
[20] 徐月霞.面向语义的数学公式N-grams索引结构研究[D].兰州: 兰州大学,2015. XU Y X.N-gram index structure for semantic based mathematical formulas[D].Lanzhou: Lanzhou University of Technology,2015.
[21] 周南.LaTeX数学表达式解析与索引方法[J].计算机应用,2016,36(3):833-836,842. ZHOU N.Analyzing and indexing method on LaTeX formulae[J].Journal of Computer Applications, 2016,36(3):833- 836,842.DOI:10.11772/j.issn.1001-9081.2016.03.833.DOI:10.11772/j.issn.1001-9081.2016.03.833.
