摘要
容器化部署Flink时,存在上下游算子的容器内存分配不均衡问题。提出基于深度学习的容器化Flink上下游负载均衡框架,使用CEEMDAN分解方法和BiLSTM相结合的预测方法预测Flink下游容器所需内存,并依据预测结果调整容器内存分配。实验证明:提出的上下游负载均衡策略可有效减少上游容器的等待时间,缓解下游容器的资源,计算效率提高约20%。
When deploying Flink in container environment,the container resources of upstream and downstream tasks can hardly be allocated balancedly.A containerized Flink upstream and downstream load balancing framework based on deep learning is proposed.The prediction method combining CEEMDAN decomposition method and BiLSTM is used to predict the memory required by Flink downstream containers.The container memory allocation is adjusted according to the prediction results.Experiments show that the proposed upstream and downstream load balancing strategy can effectively reduce the waiting time of upstream containers,alleviate the resources of downstream containers,and improve the computing efficiency by about 20%.
作者
艾力卡木·再比布拉
甄妞
黄山
段晓东
Alkam Zabibul;ZHEN Niu;HUANG Shan;DUAN Xiao-dong(School of Computer Science and Engineering,Dlian Minzu University,Dlian Liaoning 116650,China;Key Laboratory of Big Data Applied Technology of State of Ethnic Affairs Commission,Dlian Minzu University,Dlian Liaoning 116650,China;Dalian Key Laboratory of Digital Technology for National Culture,Dlian Minzu University,Dlian Liaoning 116650,China)
出处
《大连民族大学学报》
2023年第1期47-52,共6页
Journal of Dalian Minzu University
基金
国家重点研发计划云计算和大数据重点专项项目(2018YFB1004402)。
