摘要
对不同语言的句对齐文献资料进行分析,提出了基于多语主题模型的跨语言文献相似度的计算方法.首先,对收集整理的不同语言(中文、英文、韩文)文献构建数据模型,通过分词、分词结果修正及选择、词权重计算等预处理工作构造词项-文档矩阵.其次,建立多语主题语义空间,将译成3种不同语言的文献映射到语义空间,在语义空间中每一主题都由3种语言构成.最后,通过其语义空间中对应的主题计算比较不同语言间的文献相似度.实验结果显示,不同语言之间的文献相似度可以直接在语义空间中计算,且相似度计算的准确性在90%以上,验证了本文方法在跨语言文献相似度计算时的有效性.
We analyse different language literatures with sentence alignment and propose a cross lingual literatures＇ similarity method based on multilingual topic correlation model. In this paper, the data model for the collected different language literatures is firstly gained by term-document matrix, which is obtained by the process of words segmentation, the adjustment and selection of words segmentation results, and the weight calculation of feature words. And then, multilingual topic correlation semantic space is built. The three different language literatures are represented in the semantic space where each topic is made up of the three languages. Similarity calculation of different language literatures is completed by their correlation topic in the semantic space. Experiment results show that the similarity of different language literaturescan be calculated directly in the semantic space, the accuracy can be reached 90 %, which verify the effectiveness of our method in calculating the similarity of cross-lingual literatures.
出处
《延边大学学报（自然科学版）》
CAS
2016年第2期151-155,共5页
Journal of Yanbian University（Natural Science Edition）
基金
吉林省科技发展计划项目(20130101179JC-18)
吉林省公共计算平台资助
延边大学科技发展计划项目(延大科合字[2014]第16号)
关键词
多语主题模型
跨语言
语义相似度
multilingual topic correlation model
cross-lingual
semantic similarity
