In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinfor...In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes. We introduce features of the states of the original problem, and we formulate a smaller "aggregate" Markov decision problem, whose states relate to the features. We discuss properties and possible implementations of this type of aggregation, including a new approach to approximate policy iteration. In this approach the policy improvement operation combines feature-based aggregation with feature construction using deep neural networks or other calculations. We argue that the cost function of a policy may be approximated much more accurately by the nonlinear function of the features provided by aggregation, than by the linear function of the features provided by neural networkbased reinforcement learning, thereby potentially leading to more effective policy improvement.展开更多
Hydraulic systems have the characteristics of strong fault concealment,powerful nonlinear time-varying signals,and a complex vibration transmission mechanism;hence,diagnosis of these systems is a challenge.To provide ...Hydraulic systems have the characteristics of strong fault concealment,powerful nonlinear time-varying signals,and a complex vibration transmission mechanism;hence,diagnosis of these systems is a challenge.To provide accurate diagnosis results automatically,numerous studies have been carried out.Among them,signal-based methods are commonly used,which employ signal processing techniques based on the state signal used for extracting features,and further input the features into the classifier for fault recognition.However,their main deficiencies include the following:(1)The features are manually designed and thus may have a lack of objectivity.(2)For signal processing,feature extraction and pattern recognition are conducted using independent models,which cannot be jointly optimized globally.(3)The machine learning algorithms adopted by these methods have a shallow architecture,which limits their capacity to deeply mine the essential features of a fault.As a breakthrough in artificial intelligence,deep learning holds the potential to overcome such deficiencies.Based on deep learning,deep neural networks(DNNs)can automatically learn the complex nonlinear relations implied in a signal,can be globally optimized,and can obtain the high-level features of multi-dimensional data.In this paper,the main technology used in an intelligent fault diagnosis and the current research status of hydraulic system fault diagnosis are summarized and analyzed.The significant prospect of applying deep learning in the field of intelligent fault diagnosis is presented,and the main ideas,methods,and principles of several typical DNNs are described and summarized.The commonality between a fault diagnosis and other issues regarding typical pattern recognition are analyzed,and research ideas for applying DNNs for hydraulic fault diagnosis are proposed.Meanwhile,the research advantages and development trend of DNNs(both domestically and overseas)as applied to an intelligent fault diagnosis are reviewed.Furthermore,the fault characteristics of a c展开更多
Ti-2 Al-9.2 Mo-2 Fe is a low-cost β titanium alloy with well-balanced strength and ductility, but hot working of this alloy is complex and unfamiliar. Understanding the nonlinear relationships among the strain,strain...Ti-2 Al-9.2 Mo-2 Fe is a low-cost β titanium alloy with well-balanced strength and ductility, but hot working of this alloy is complex and unfamiliar. Understanding the nonlinear relationships among the strain,strain rate, temperature, and flow stress of this alloy is essential to optimize the hot working process.In this study, a deep neural network(DNN) model was developed to correlate flow stress with a wide range of strains(0.025–0.6), strain rates(0.01–10 s^-1) and temperatures(750–1000℃). The model, which was tested with 96 unseen datasets, showed better performance than existing models, with a correlation coefficient of 0.999. The processing map constructed using the DNN model was effective in predicting the microstructural evolution of the alloy. Moreover, it led to the optimization of hot-working conditions to avoid the formation of brittle precipitates(temperatures of 820–1000℃ and strain rates of 0.01-0.1 s^-1).展开更多
文摘In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes. We introduce features of the states of the original problem, and we formulate a smaller "aggregate" Markov decision problem, whose states relate to the features. We discuss properties and possible implementations of this type of aggregation, including a new approach to approximate policy iteration. In this approach the policy improvement operation combines feature-based aggregation with feature construction using deep neural networks or other calculations. We argue that the cost function of a policy may be approximated much more accurately by the nonlinear function of the features provided by aggregation, than by the linear function of the features provided by neural networkbased reinforcement learning, thereby potentially leading to more effective policy improvement.
基金Supported by National Natural Science Foundation of China(Grant No.51705531)Jiangsu Provincial Natural Science Foundation of China(Grant No.BK20150724)
文摘Hydraulic systems have the characteristics of strong fault concealment,powerful nonlinear time-varying signals,and a complex vibration transmission mechanism;hence,diagnosis of these systems is a challenge.To provide accurate diagnosis results automatically,numerous studies have been carried out.Among them,signal-based methods are commonly used,which employ signal processing techniques based on the state signal used for extracting features,and further input the features into the classifier for fault recognition.However,their main deficiencies include the following:(1)The features are manually designed and thus may have a lack of objectivity.(2)For signal processing,feature extraction and pattern recognition are conducted using independent models,which cannot be jointly optimized globally.(3)The machine learning algorithms adopted by these methods have a shallow architecture,which limits their capacity to deeply mine the essential features of a fault.As a breakthrough in artificial intelligence,deep learning holds the potential to overcome such deficiencies.Based on deep learning,deep neural networks(DNNs)can automatically learn the complex nonlinear relations implied in a signal,can be globally optimized,and can obtain the high-level features of multi-dimensional data.In this paper,the main technology used in an intelligent fault diagnosis and the current research status of hydraulic system fault diagnosis are summarized and analyzed.The significant prospect of applying deep learning in the field of intelligent fault diagnosis is presented,and the main ideas,methods,and principles of several typical DNNs are described and summarized.The commonality between a fault diagnosis and other issues regarding typical pattern recognition are analyzed,and research ideas for applying DNNs for hydraulic fault diagnosis are proposed.Meanwhile,the research advantages and development trend of DNNs(both domestically and overseas)as applied to an intelligent fault diagnosis are reviewed.Furthermore,the fault characteristics of a c
基金supported by grants from the Civil–Military Technology Cooperation Program (16-CM-MA-10) of the Defense Acquisition Program Administrationfrom the Core Material Program (10062485) of the Ministry of Trade, Industry and Energy, Republic of Korea
文摘Ti-2 Al-9.2 Mo-2 Fe is a low-cost β titanium alloy with well-balanced strength and ductility, but hot working of this alloy is complex and unfamiliar. Understanding the nonlinear relationships among the strain,strain rate, temperature, and flow stress of this alloy is essential to optimize the hot working process.In this study, a deep neural network(DNN) model was developed to correlate flow stress with a wide range of strains(0.025–0.6), strain rates(0.01–10 s^-1) and temperatures(750–1000℃). The model, which was tested with 96 unseen datasets, showed better performance than existing models, with a correlation coefficient of 0.999. The processing map constructed using the DNN model was effective in predicting the microstructural evolution of the alloy. Moreover, it led to the optimization of hot-working conditions to avoid the formation of brittle precipitates(temperatures of 820–1000℃ and strain rates of 0.01-0.1 s^-1).
