摘要
Hadoop具有海量数据并行存储能力和高效并行计算架构,但缺乏数据建模和数据统计能力.针对Hadoop架构的数据统计分析能力的局限性,结合R语言和Hadoop框架的优点,提出一种基于R+Hadoop环境的大数据分析及预测方法.以甘肃惠森药业电子商务平台"药材盈"采集的大数据为例,通过采用Hadoop集群并行处理中药材文本数据、RHadoop进行预处理并获取样本数据、R语言对样本数据建模,获得较为可靠的预测中药材市场价格的模型,对中药材市场价格的变化规律及影响因素进行分析和预测.采用线性模型和决策树模型对中药材大数据进行建模,并通过实验验证和比较得到预测中药材市场价格的最佳模型.
Hadoop has parallel storage capacity and efficient parallel computing architecture for massive data, but lacks ability of data modeling and data statistics. Aimed at the limitations of data statistic analysis ability of the Hadoop framework, a method for big data analysis and forecasting is proposed by means of combining the advantages of R language and Hadoop framework in R＋ Hadoop environment. Taking the big data gathered by ＂herbal surplus＂ on e-commerce platform Gansu of Huisen pharmacal industry as an example and using Hadoop cluster for parallel processing of Chinese herbal medicine text data, RHadoop for preprocessing and getting sample data, and R language for modeling of the sample data, a more reliable model for predicting the market price of Chinese medicinal materials is obtained and the variation pattern and influencing factor of the market price of Chinese medicinal materials are analyzed and predicted. By using linear model and decision tree model, the modeling of big data is conducted for Chinese medicinal materials and by means of experiment and comparison, the optimal model for prediction of market price of Chinese medicinal materials is obtained.
出处
《兰州理工大学学报》
CAS
北大核心
2017年第1期98-103,共6页
Journal of Lanzhou University of Technology
基金
兰州理工大学红柳人才计划项目(J201304)的资助
关键词
中药材
R语言
数据建模
Hadoop技术
决策树
Chinese medicinal materials
R language
data modeling
Hadoop technique
decision tree
