摘要
本文针对企业网络安全日志分析场景,采用实际网络安全日志数据,设计建表和测试方法,使用大数据引擎Spark SQL和Presto对索引行列混合式数据格式CarbonData进行了性能评估。通过测试得出,基于Spark SQL+CarbonData方案的数据查询和统计效率较高,能满足多种数据处理和分析要求,适用于网络安全日志分析应用。本文为企业网络安全日志分析方案的技术选型和设计提供参考。
This article focuses on the analysis scenario of enterprise network security logs,using actual network security log data,designing table building and testing methods,and using big data engines Spark SQL and Presto to evaluate the performance of the indexed columnar data format CarbonData.Through testing,it has been found that the data query and statistical efficiency based on Spark SQL+CarbonData scheme is high,which can meet various data processing and analysis requirements and is suitable for network security log analysis applications.This article provides a reference for the technical selection and design of enterprise network security log analysis solutions.
作者
黄必栋
Huang Bidong(School of Intelligent Engineering,Nanjing Vocational Institute of Railway Technology,Nanjing,China)
出处
《科学技术创新》
2024年第5期99-105,共7页
Scientific and Technological Innovation
