摘要
对大数据库的相似记录进行准确检测,可以检测提高数据库重复记录检测效果。对数据库中非重要数据进行删除,对剩余数据属性进行分类是相似记录检测的重要前提,但是传统方法量子粒子群算法只能对整体数据相似度特征样本进行训练,不能消除数据库中的非重要数据,增加了检测时间,降低了相似记录检测的效率。提出改进二次模糊评判的大数据库的相似记录检测方法。上述方法先依据等级法对大数据库中全部数据属性进行第一次评判,消除数据库中非重要数据属性,对剩余数据属性进行二次模糊评判,并均衡其数据属性的评判结果,对数据库的全部数据属性进行分组,在此基础上计算出各组数据集的记录字段间的相似度函数,提取记录字段相似度的特征向量,组建大数据库的相似记录检测模型,完成对大数据库的相似记录检测。仿真结果表明,所提算法可以有效地提升大数据库的相似记录检测的查准率。
Traditional algorithm of quantum particle swarm only trains similarity feature sample of whole data.It cannot eliminate non-important data in database,which increases detection time and reduces efficiency.Therefore,this paper provides a test method of similar record of large database based on modified second fuzzy evaluation.In the proposed method,we judge all data attributes in the large database for the first time according to rank method and eliminated the non-important data.The second fuzzy evaluation is carried out to residual data attribute.We also balance its evaluation results;then group all data attributes in database and work out similarity function between record fields of each dataset on that basis.Additionally,we extract characteristic vector of similarity of record fields and build detection model to complete the similar record test.Simulation results indicate that the method can improve precision ratio of similar record test effectively.
出处
《计算机仿真》
北大核心
2017年第3期410-413,共4页
Computer Simulation
关键词
大数据库
相似记录
检测
Large database
Similar record
Detection
