The present study investigates an energy management strategy based on reinforcement learning for seriesparallel hybrid vehicles. Hybrid electric vehicles allow using more advanced power management policies because of ...The present study investigates an energy management strategy based on reinforcement learning for seriesparallel hybrid vehicles. Hybrid electric vehicles allow using more advanced power management policies because of their complexity of power management. Towards this feature, a Q-Learning algorithm is proposed to design an energy management strategy. Compared to previous studies, an online reward function is defined to optimize fuel consumption and battery life cycle. Moreover, in the provided method, prior knowledge of the cycle and exact modeling of the vehicle are not required. The introduced strategy is simulated for four driving cycles in MATLAB software linked with ADVISOR. The simulation results show that in the HWFET cycle, the fuel consumption decreases by 1.25 %, and battery life increases by 65% compared to the rule-based method implemented in ADVISOR. Also, the results for the other driving cycles confirm the self-improvement property. In addition, it has been depicted that in the case of change in the driving cycle, the method performance has been maintained and gained better performance than the rule-based controller.展开更多
To realize high-accuracy physical-cyber digital twin(DT)mapping in a manufacturing system,a huge amount of data need to be collected and analyzed in real-time.Traditional DTs systems are deployed in cloud or edge serv...To realize high-accuracy physical-cyber digital twin(DT)mapping in a manufacturing system,a huge amount of data need to be collected and analyzed in real-time.Traditional DTs systems are deployed in cloud or edge servers independently,whilst it is hard to apply in real production systems due to the high interaction or execution delay.This results in a low consistency in the temporal dimension of the physical-cyber model.In this work,we propose a novel efficient edge-cloud DT manufacturing system,which is inspired by resource scheduling technology.Specifically,an edge-cloud collaborative DTs system deployment architecture is first constructed.Then,deterministic and uncertainty optimization adaptive strategies are presented to choose a more powerful server for running DT-based applications.We model the adaptive optimization problems as dynamic programming problems and propose a novel collaborative clustering parallel Q-learning(CCPQL)algorithm and prediction-based CCPQL to solve the problems.The proposed approach reduces the total delay with a higher convergence rate.Numerical simulation results are provided to validate the approach,which would have great potential in dynamic and complex industrial internet environments.展开更多
The emerging development of connected and automated vehicles imposes a significant challenge on current vehicle control and transportation systems. This paper proposes a novel unified approach, Parallel Driving, a clo...The emerging development of connected and automated vehicles imposes a significant challenge on current vehicle control and transportation systems. This paper proposes a novel unified approach, Parallel Driving, a cloud-based cyberphysical-social systems(CPSS) framework aiming at synergizing connected automated driving. This study first introduces the CPSS and ACP-based intelligent machine systems. Then the parallel driving is proposed in the cyber-physical-social space,considering interactions among vehicles, human drivers, and information. Within the framework, parallel testing, parallel learning and parallel reinforcement learning are developed and concisely reviewed. Development on intelligent horizon(iHorizon)and its applications are also presented towards parallel horizon.The proposed parallel driving offers an ample solution for achieving a smooth, safe and efficient cooperation among connected automated vehicles with different levels of automation in future road transportation systems.展开更多
As a complex and critical cyber-physical system(CPS),the hybrid electric powertrain is significant to mitigate air pollution and improve fuel economy.Energy management strategy(EMS)is playing a key role to improve the...As a complex and critical cyber-physical system(CPS),the hybrid electric powertrain is significant to mitigate air pollution and improve fuel economy.Energy management strategy(EMS)is playing a key role to improve the energy efficiency of this CPS.This paper presents a novel bidirectional long shortterm memory(LSTM)network based parallel reinforcement learning(PRL)approach to construct EMS for a hybrid tracked vehicle(HTV).This method contains two levels.The high-level establishes a parallel system first,which includes a real powertrain system and an artificial system.Then,the synthesized data from this parallel system is trained by a bidirectional LSTM network.The lower-level determines the optimal EMS using the trained action state function in the model-free reinforcement learning(RL)framework.PRL is a fully data-driven and learning-enabled approach that does not depend on any prediction and predefined rules.Finally,real vehicle testing is implemented and relevant experiment data is collected and calibrated.Experimental results validate that the proposed EMS can achieve considerable energy efficiency improvement by comparing with the conventional RL approach and deep RL.展开更多
基金Supported by National Natural Science Foundation of China（60474035）,National Research Foundation for the Doctoral Program of Higher Education of China（20050359004）,Natural Science Foundation of Anhui Province（070412035）
文摘The present study investigates an energy management strategy based on reinforcement learning for seriesparallel hybrid vehicles. Hybrid electric vehicles allow using more advanced power management policies because of their complexity of power management. Towards this feature, a Q-Learning algorithm is proposed to design an energy management strategy. Compared to previous studies, an online reward function is defined to optimize fuel consumption and battery life cycle. Moreover, in the provided method, prior knowledge of the cycle and exact modeling of the vehicle are not required. The introduced strategy is simulated for four driving cycles in MATLAB software linked with ADVISOR. The simulation results show that in the HWFET cycle, the fuel consumption decreases by 1.25 %, and battery life increases by 65% compared to the rule-based method implemented in ADVISOR. Also, the results for the other driving cycles confirm the self-improvement property. In addition, it has been depicted that in the case of change in the driving cycle, the method performance has been maintained and gained better performance than the rule-based controller.
基金supported by 2019 Industrial Internet Innovation Development Project of Ministry of Industry and Information Technology of P.R. China “Comprehensive Security Defense Platform Project for Industrial/Enterprise Networks”Research on Key Technologies of wireless edge intelligent collaboration for industrial internet scenarios (L202017)+1 种基金Natural Science Foundation of China, No.61971050BUPT Excellent Ph.D. Students Foundation (CX2020214)。
文摘To realize high-accuracy physical-cyber digital twin(DT)mapping in a manufacturing system,a huge amount of data need to be collected and analyzed in real-time.Traditional DTs systems are deployed in cloud or edge servers independently,whilst it is hard to apply in real production systems due to the high interaction or execution delay.This results in a low consistency in the temporal dimension of the physical-cyber model.In this work,we propose a novel efficient edge-cloud DT manufacturing system,which is inspired by resource scheduling technology.Specifically,an edge-cloud collaborative DTs system deployment architecture is first constructed.Then,deterministic and uncertainty optimization adaptive strategies are presented to choose a more powerful server for running DT-based applications.We model the adaptive optimization problems as dynamic programming problems and propose a novel collaborative clustering parallel Q-learning(CCPQL)algorithm and prediction-based CCPQL to solve the problems.The proposed approach reduces the total delay with a higher convergence rate.Numerical simulation results are provided to validate the approach,which would have great potential in dynamic and complex industrial internet environments.
文摘The emerging development of connected and automated vehicles imposes a significant challenge on current vehicle control and transportation systems. This paper proposes a novel unified approach, Parallel Driving, a cloud-based cyberphysical-social systems(CPSS) framework aiming at synergizing connected automated driving. This study first introduces the CPSS and ACP-based intelligent machine systems. Then the parallel driving is proposed in the cyber-physical-social space,considering interactions among vehicles, human drivers, and information. Within the framework, parallel testing, parallel learning and parallel reinforcement learning are developed and concisely reviewed. Development on intelligent horizon(iHorizon)and its applications are also presented towards parallel horizon.The proposed parallel driving offers an ample solution for achieving a smooth, safe and efficient cooperation among connected automated vehicles with different levels of automation in future road transportation systems.
基金supported in part by the National Natural Science Foundation of China(61533019,91720000)Beijing Municipal Science and Technology Commission(Z181100008918007)the Intel Collaborative Research Institute for Intelligent and Automated Connected Vehicles(pICRI-IACVq)
文摘As a complex and critical cyber-physical system(CPS),the hybrid electric powertrain is significant to mitigate air pollution and improve fuel economy.Energy management strategy(EMS)is playing a key role to improve the energy efficiency of this CPS.This paper presents a novel bidirectional long shortterm memory(LSTM)network based parallel reinforcement learning(PRL)approach to construct EMS for a hybrid tracked vehicle(HTV).This method contains two levels.The high-level establishes a parallel system first,which includes a real powertrain system and an artificial system.Then,the synthesized data from this parallel system is trained by a bidirectional LSTM network.The lower-level determines the optimal EMS using the trained action state function in the model-free reinforcement learning(RL)framework.PRL is a fully data-driven and learning-enabled approach that does not depend on any prediction and predefined rules.Finally,real vehicle testing is implemented and relevant experiment data is collected and calibrated.Experimental results validate that the proposed EMS can achieve considerable energy efficiency improvement by comparing with the conventional RL approach and deep RL.
