摘要
缺失数据的处理是数据清洗的重要内容。提出了一种基于位运算的不完整记录分类检测方法。对不完整记录进行了界定,将记录分为完整、不完整合格、不完整修正和不完整删除四类,并给出了其层次分类流程。定义了记录的二进制表示,根据不完整记录样本生成各类记录的标准二进制表示集,按在样本中出现的次数确定标准二进制表示的优先级,并对不完整删除标准二进制表示集中的二制表示进行了表达式合并。通过位运算实现记录的分类检测,并通过处理未检出二进制表示逐步完善二进制表示集。根据不完整记录二进制表示确定记录的进一步处理。应用实例验证了方法的有效性。
Missing data treatment is an important content of data cleaning.A classification detection method for uncompleted records is proposed.The uncompleted record is defined and records are classified as four classes,including completed records,uncompleted and unmodifying records,uncompleted and modifying records,uncompleted and deleting records.A classifying flow with hiberarchy is given.The binary expression of a record is defined.The standard binary expression sets of each class are created according to uncompleted record samples.Priority of standard binary expressions is determined by appearance times in samples.Some specific binary expressions are merged using formulas.Classification detection of records is implemented by bit operation.Binary expression sets are perfected step by step through dealing unseen binary expressions.The next processing of uncompleted records could be confirmed by their binary expressions.The effectiveness of the proposed method is validated by an instance.
出处
《系统工程与电子技术》
EI
CSCD
北大核心
2010年第11期2489-2492,共4页
Systems Engineering and Electronics
基金
中国博士后科学基金(20090461425)
江苏省博士后科研资助计划项目(0901014B)资助课题
关键词
数据质量
数据清洗
缺失数据
不完整记录
分类
data quality
data cleaning
missing data
incompleted record
classification
