摘要
针对智能网络大数据分布正则性差的问题,提出基于动态可重用性结构化分区融合的大数据清洗规则链自动生成方法。先基于动态配置网络构建规则类型分布集,采用动态编译方法实现对数据的语义特征检测和稀疏参数辨识,采用多维关系网络分组检测方法进行数据的局部谱密度聚类。建立数据规则链的实体结构模型,通过数据聚类的张量表达实现动态可重用性结构化分区处理和信息融合,通过网络大数据的多维尺度扩展聚类处理,实现对干扰数据的分组滤波,基于链路的聚类方法实现对网络大数据的结构化数据重排,实现智能网络大数据清洗规则链自动生成。仿真测试结果表明,采用该方法进行网络大数据清洗的抗干扰性较好,清洗的准确回填性高于95%,且100条数据的平均清洗用时为12.6 ms,性能优于对比方法,提高了网络大数据信息提取和辨识能力,具有较大的应用价值。
Aiming at the problem of poor distribution regularity of big data in intelligent network, an automatic generation method of big data cleaning rule chain based on dynamic reusability structured partition fusion is proposed. The distribution set of rule types is buict based on the dynamic configuration network, the dynamic compilation method of rules is used to realize the semantic feature detection and sparse parameter identification of the data, and the multi-dimensional relational network grouping detection method is used to perform the data local spectral density clustering. A entity structure model is established connected by data rules to realize dynamic reusability and structure partition processing and information fusion through the tensor expression of data clustering, and achieve group filtering of interference data through multi-dimensional scale expansion of network big data clustering processing, The link-based clustering method realizes the structured data rearrangement of the network big data, and realizes the automatic generation of the intelligent network big data cleaning rule chain. Simulation test results show that using this method for network big data cleaning has better anti-interference, the accuracy of cleaning backfill is higher than 95%, and the average cleaning time of 100 data is 12.6 ms, which is better than the comparison method and improves The ability of network big data information extraction and identification has great application value.
作者
潘海霞
曹宁
PAN Haixia;CAO Ning(Shaanxi Police College,Xi’an 710021,China;Logistics Management Department of Northwest A&F University,Xianyang City,Shaanxi 712100,China)
出处
《自动化与仪器仪表》
2022年第9期58-61,65,共5页
Automation & Instrumentation
基金
陕西省职业技术教育学会支持项目(2022SZX237)。
关键词
动态配置网络
大数据
清洗
规则链
动态编译
dynamic configuration network
big data
cleaning
rules
dynamic compilation
