作为流式大数据计算的主要平台之一,Storm在设计过程中由于缺乏节能的考虑,导致其存在高能耗与低效率的问题.传统的节能策略并未考虑Storm的性能约束,可能会对集群的实时性造成影响.针对这一问题,设计了资源约束模型、最优线程重分配模...作为流式大数据计算的主要平台之一,Storm在设计过程中由于缺乏节能的考虑,导致其存在高能耗与低效率的问题.传统的节能策略并未考虑Storm的性能约束,可能会对集群的实时性造成影响.针对这一问题,设计了资源约束模型、最优线程重分配模型以及数据迁移模型.进一步提出了Storm平台下的线程重分配与数据迁移节能策略(energy-efficient strategy based on executor reallocation and data migration in Storm,简称ERDM),包括资源约束算法与数据迁移算法.其中,资源约束算法根据集群各工作节点CPU、内存与网络带宽的资源占用率,判断集群是否允许数据的迁移.数据迁移算法根据资源约束模型与最优线程重分配模型,设计了数据迁移的最优化方法.此外,ERDM通过分配线程减少了节点间的通信开销,并根据大数据流式计算的性能与能效评估ERDM.实验结果表明,与现有研究相比,ERDM能够有效降低节点间通信开销与能耗,并提高集群的性能.展开更多
This paper explores the opportunities and challenges of college mental health education from the perspective of big data.Firstly,through literature review,the importance of mental health education and the current issu...This paper explores the opportunities and challenges of college mental health education from the perspective of big data.Firstly,through literature review,the importance of mental health education and the current issues are elucidated.Then,from the perspective of big data,the potential opportunities of big data in college mental health education are analyzed,including data-driven personalized education,real-time monitoring and warning systems,and interdisciplinary research and collaboration.At the same time,the challenges faced by college mental health education under the perspective of big data are also pointed out,such as data privacy and security issues,insufficient data analysis and interpretation capabilities,and inadequate technical facilities and talent support.Lastly,the research content of this paper is summarized,and directions and suggestions for future research are proposed.展开更多
With the advent of Big Data, the fields of Statistics and Computer Science coexist in current information systems. In addition to this, technological advances in embedded systems, in particular Internet of Things tech...With the advent of Big Data, the fields of Statistics and Computer Science coexist in current information systems. In addition to this, technological advances in embedded systems, in particular Internet of Things technologies, make it possible to develop real-time applications. These technological developments are disrupting Software Engineering because the use of large amounts of real-time data requires advanced thinking in terms of software architecture. The purpose of this article is to propose an architecture unifying not only Software Engineering and Big Data activities, but also batch and streaming architectures for the exploitation of massive data. This architecture has the advantage of making possible the development of applications and digital services exploiting very large volumes of data in real time;both for management needs and for analytical purposes. This architecture was tested on COVID-19 data as part of the development of an application for real-time monitoring of the evolution of the pandemic in Côte d’Ivoire using PostgreSQL, ELasticsearch, Kafka, Kafka Connect, NiFi, Spark, Node-Red and MoleculerJS to operationalize the architecture.展开更多
The outputs of a national economy can be partitioned into three sets of products:tangible goods(due to manufacturing,construction,extraction and agriculture),intangible services(due to an act of useful effort),and an ...The outputs of a national economy can be partitioned into three sets of products:tangible goods(due to manufacturing,construction,extraction and agriculture),intangible services(due to an act of useful effort),and an integration of services and goods or,as initially defined by Tien(2012),servgoods.Actually,these products can also be considered in terms of their relation to the first three Industrial Revolutions:the First Industrial Revolution(circa 1800)was primarily focused on the production of goods;the Second Industrial Revolution(circa 1900)was primarily focused on the mass production of goods;and the Third Industrial Revolution(circa 2000)has been primarily focused on the mass customization of goods,services or servgoods.In this follow-up paper,the Third Industrial Revolution of mass customization continues to accelerate in its evolution and,in many respects,is subsuming the earlier Industrial Revolutions of production and mass production.More importantly,with the advent of real-time decision making,artificial intelligence,Internet of Things,mobile networks,and other advanced digital technologies,customization has been extensively enabled,thereby advancing mass customization into a Fourth Industrial Revolution of real-time customization.Moreover,the moral,ethical,security and employment problems associated with both mass and real-time customization must be carefully assessed and mitigated,especially in regard to unintended consequences.Looking ahead and with the advance of artificial general intelligence,this Fourth Industrial Revolution could be forthcoming in about the middle of the 21st Century;it would allow for multiple activities to be simultaneously tackled in real-time and in a customized manner.展开更多
文摘作为流式大数据计算的主要平台之一,Storm在设计过程中由于缺乏节能的考虑,导致其存在高能耗与低效率的问题.传统的节能策略并未考虑Storm的性能约束,可能会对集群的实时性造成影响.针对这一问题,设计了资源约束模型、最优线程重分配模型以及数据迁移模型.进一步提出了Storm平台下的线程重分配与数据迁移节能策略(energy-efficient strategy based on executor reallocation and data migration in Storm,简称ERDM),包括资源约束算法与数据迁移算法.其中,资源约束算法根据集群各工作节点CPU、内存与网络带宽的资源占用率,判断集群是否允许数据的迁移.数据迁移算法根据资源约束模型与最优线程重分配模型,设计了数据迁移的最优化方法.此外,ERDM通过分配线程减少了节点间的通信开销,并根据大数据流式计算的性能与能效评估ERDM.实验结果表明,与现有研究相比,ERDM能够有效降低节点间通信开销与能耗,并提高集群的性能.
文摘This paper explores the opportunities and challenges of college mental health education from the perspective of big data.Firstly,through literature review,the importance of mental health education and the current issues are elucidated.Then,from the perspective of big data,the potential opportunities of big data in college mental health education are analyzed,including data-driven personalized education,real-time monitoring and warning systems,and interdisciplinary research and collaboration.At the same time,the challenges faced by college mental health education under the perspective of big data are also pointed out,such as data privacy and security issues,insufficient data analysis and interpretation capabilities,and inadequate technical facilities and talent support.Lastly,the research content of this paper is summarized,and directions and suggestions for future research are proposed.
文摘With the advent of Big Data, the fields of Statistics and Computer Science coexist in current information systems. In addition to this, technological advances in embedded systems, in particular Internet of Things technologies, make it possible to develop real-time applications. These technological developments are disrupting Software Engineering because the use of large amounts of real-time data requires advanced thinking in terms of software architecture. The purpose of this article is to propose an architecture unifying not only Software Engineering and Big Data activities, but also batch and streaming architectures for the exploitation of massive data. This architecture has the advantage of making possible the development of applications and digital services exploiting very large volumes of data in real time;both for management needs and for analytical purposes. This architecture was tested on COVID-19 data as part of the development of an application for real-time monitoring of the evolution of the pandemic in Côte d’Ivoire using PostgreSQL, ELasticsearch, Kafka, Kafka Connect, NiFi, Spark, Node-Red and MoleculerJS to operationalize the architecture.
文摘The outputs of a national economy can be partitioned into three sets of products:tangible goods(due to manufacturing,construction,extraction and agriculture),intangible services(due to an act of useful effort),and an integration of services and goods or,as initially defined by Tien(2012),servgoods.Actually,these products can also be considered in terms of their relation to the first three Industrial Revolutions:the First Industrial Revolution(circa 1800)was primarily focused on the production of goods;the Second Industrial Revolution(circa 1900)was primarily focused on the mass production of goods;and the Third Industrial Revolution(circa 2000)has been primarily focused on the mass customization of goods,services or servgoods.In this follow-up paper,the Third Industrial Revolution of mass customization continues to accelerate in its evolution and,in many respects,is subsuming the earlier Industrial Revolutions of production and mass production.More importantly,with the advent of real-time decision making,artificial intelligence,Internet of Things,mobile networks,and other advanced digital technologies,customization has been extensively enabled,thereby advancing mass customization into a Fourth Industrial Revolution of real-time customization.Moreover,the moral,ethical,security and employment problems associated with both mass and real-time customization must be carefully assessed and mitigated,especially in regard to unintended consequences.Looking ahead and with the advance of artificial general intelligence,this Fourth Industrial Revolution could be forthcoming in about the middle of the 21st Century;it would allow for multiple activities to be simultaneously tackled in real-time and in a customized manner.
