Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicabilit...Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.展开更多
Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularl...Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.展开更多
Emergence of new hardware,including persistent memory and smart network interface card(SmartNIC),has brought new opportunities to file system design.In this paper,we design and implement a new file system named NICFS ...Emergence of new hardware,including persistent memory and smart network interface card(SmartNIC),has brought new opportunities to file system design.In this paper,we design and implement a new file system named NICFS based on persistent memory and SmartNIC.We divide the file system into two parts:the front end and the back end.In the front end,data writes are appended to the persistent memory in a log-structured way,leveraging the fast persistence advantage of persistent memory.In the back end,the data in logs are fetched,processed,and patched to files in the background,leveraging the processing capacity of SmartNIC.Evaluation results show that NICFS outperforms Ext4 by about 21%/10%and about 19%/50%on large and small reads/writes,respectively.展开更多
This paper presents designing sequence-to-sequence recurrent neural network(RNN)architectures for a novel study to predict soil NOx emissions,driven by the imperative of understanding and mitigating environmental impa...This paper presents designing sequence-to-sequence recurrent neural network(RNN)architectures for a novel study to predict soil NOx emissions,driven by the imperative of understanding and mitigating environmental impact.The study utilizes data collected by the Environmental Protection Agency(EPA)to develop two distinct RNN predictive models:one built upon the long-short term memory(LSTM)and the other utilizing the gated recurrent unit(GRU).These models are fed with a combination of historical and anticipated air temperature,air moisture,and NOx emissions as inputs to forecast future NOx emissions.Both LSTM and GRU models can capture the intricate pulse patterns inherent in soil NOx emissions.Notably,the GRU model emerges as the superior performer,surpassing the LSTM model in predictive accuracy while demonstrating efficiency by necessitating less training time.Intriguingly,the investigation into varying input features reveals that relying solely on past NOx emissions as input yields satisfactory performance,highlighting the dominant influence of this factor.The study also delves into the impact of altering input series lengths and training data sizes,yielding insights into optimal configurations for enhanced model performance.Importantly,the findings promise to advance our grasp of soil NOx emission dynamics,with implications for environmental management strategies.Looking ahead,the anticipated availability of additional measurements is poised to bolster machine-learning model efficacy.Furthermore,the future study will explore physical-based RNNs,a promising avenue for deeper insights into soil NOx emission prediction.展开更多
Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to ...Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to the strict time serial dependency,making it difficult to realize a long-term memory function.On the other hand,RNNs cells are highly complex,which will signifi-cantly increase computational complexity and cause waste of computational resources during model training.In this paper,an improved Time Feedforward Connections Recurrent Neural Networks(TFC-RNNs)model was first proposed to address the gradient issue.A parallel branch was introduced for the hidden state at time t−2 to be directly transferred to time t without the nonlinear transforma-tion at time t−1.This is effective in improving the long-term dependence of RNNs.Then,a novel cell structure named Single Gate Recurrent Unit(SGRU)was presented.This cell structure can reduce the number of parameters for RNNs cell,consequently reducing the computational complexity.Next,applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the above two difficulties.Finally,the performance of our proposed TFC-SGRU was verified through sev-eral experiments in terms of long-term memory and anti-interference capabilities.Experimental results demonstrated that our proposed TFC-SGRU model can cap-ture helpful information with time step 1500 and effectively filter out the noise.The TFC-SGRU model accuracy is better than the LSTM and GRU models regarding language processing ability.展开更多
Lithium-ion batteries are commonly used in electric vehicles,mobile phones,and laptops.These batteries demonstrate several advantages,such as environmental friendliness,high energy density,and long life.However,batter...Lithium-ion batteries are commonly used in electric vehicles,mobile phones,and laptops.These batteries demonstrate several advantages,such as environmental friendliness,high energy density,and long life.However,battery overcharging and overdischarging may occur if the batteries are not monitored continuously.Overcharging causesfire and explosion casualties,and overdischar-ging causes a reduction in the battery capacity and life.In addition,the internal resistance of such batteries varies depending on their external temperature,elec-trolyte,cathode material,and other factors;the capacity of the batteries decreases with temperature.In this study,we develop a method for estimating the state of charge(SOC)using a neural network model that is best suited to the external tem-perature of such batteries based on their characteristics.During our simulation,we acquired data at temperatures of 25°C,30°C,35°C,and 40°C.Based on the tem-perature parameters,the voltage,current,and time parameters were obtained,and six cycles of the parameters based on the temperature were used for the experi-ment.Experimental data to verify the proposed method were obtained through a discharge experiment conducted using a vehicle driving simulator.The experi-mental data were provided as inputs to three types of neural network models:mul-tilayer neural network(MNN),long short-term memory(LSTM),and gated recurrent unit(GRU).The neural network models were trained and optimized for the specific temperatures measured during the experiment,and the SOC was estimated by selecting the most suitable model for each temperature.The experimental results revealed that the mean absolute errors of the MNN,LSTM,and GRU using the proposed method were 2.17%,2.19%,and 2.15%,respec-tively,which are better than those of the conventional method(4.47%,4.60%,and 4.40%).Finally,SOC estimation based on GRU using the proposed method was found to be 2.15%,which was the most accurate.展开更多
The monitoring of Li-ion battery temperatures is essential to ensure high efficiency and safety.In this work,two types of recurrent neural networks (RNNs),which are long short-term memory-RNN (LSTM-RNN) and gated recu...The monitoring of Li-ion battery temperatures is essential to ensure high efficiency and safety.In this work,two types of recurrent neural networks (RNNs),which are long short-term memory-RNN (LSTM-RNN) and gated recurrent unit-RNN(GRU-RNN),are proposed to estimate the surface temperature of 18650 Li-ion batteries during the discharging processes under different ambient temperatures.The datasets acquired from the Prognostics Center of Excellence (PCo E) of NASA are used to train,validate and test the networks.In previous work,temperature has been set as the output of the networks;however,here the temperature difference along the time axis is adopted as the output.The net heat generated results in net temperature change,which is more closely aligned with electrochemical and thermodynamic laws.Extensive simulation results show that the two RNNs can achieve accurate real-time battery temperature estimation.The maximum absolute error in temperature estimation is approximately 0.75°C and the correlation coefficient between the estimated and measured temperature curves is greater than 0.95.The influences of three crucial parameters,which are the number of hidden neurons,initial learning rate and maximum number of iterations,are also assessed in terms of training time,root mean square error and mean absolute error.展开更多
Computer-empowered detection of possible faults for Heating,Ventilation and Air-Conditioning(HVAC)subsystems,e.g.,chillers,is one of the most important applications in Artificial Intelligence(AI)integrated Internet of...Computer-empowered detection of possible faults for Heating,Ventilation and Air-Conditioning(HVAC)subsystems,e.g.,chillers,is one of the most important applications in Artificial Intelligence(AI)integrated Internet of Things(IoT).The cyber-physical system greatly enhances the safety and security of the working facilities,reducing time,saving energy and protecting humans’health.Under the current trends of smart building design and energy management optimization,Automated Fault Detection and Diagnosis(AFDD)of chillers integrated with IoT is highly demanded.Recent studies show that standard machine learning techniques,such as Principal Component Analysis(PCA),Support Vector Machine(SVM)and tree-structure-based algorithms,are useful in capturing various chiller faults with high accuracy rates.With the fast development of deep learning technology,Convolutional Neural Networks(CNNs)have been widely and successfully applied to various fields.However,for chiller AFDD,few existing works are adopting CNN and its extensions in the feature extraction and classification processes.In this study,we propose to perform chiller FDD using a CNN-based approach.The proposed approach has two distinct advantages over existing machine learning-based chiller AFDD methods.First,the CNN-based approach does not require the feature selection/extraction process.Since CNN is reputable with its feature extraction capability,the feature extraction and classification processes are merged,leading to a more neat AFDD framework compared to traditional approaches.Second,the classification accuracy is significantly improved compared to traditional methods using the CNN-based approach.展开更多
基金supported by the Natural Science Foundation of China(Grant Nos.51979158,51639008,51679135,and 51422905)the Program of Shanghai Academic Research Leader by Science and Technology Commission of Shanghai Municipality(Project No.19XD1421900)。
文摘Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.
文摘Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.
基金Project supported by the National Key R&D Program of China(No.2021YFB0300500)the National Natural Science Foundation of China(No.62022051)。
文摘Emergence of new hardware,including persistent memory and smart network interface card(SmartNIC),has brought new opportunities to file system design.In this paper,we design and implement a new file system named NICFS based on persistent memory and SmartNIC.We divide the file system into two parts:the front end and the back end.In the front end,data writes are appended to the persistent memory in a log-structured way,leveraging the fast persistence advantage of persistent memory.In the back end,the data in logs are fetched,processed,and patched to files in the background,leveraging the processing capacity of SmartNIC.Evaluation results show that NICFS outperforms Ext4 by about 21%/10%and about 19%/50%on large and small reads/writes,respectively.
基金support from the University of Iowa Jumpstarting Tomorrow Community Feasibility Grants and OVPR Interdisciplinary Scholars Program for this study.Z.Wang and S.Xiao received support from the U.S.Department of Education(E.D.#P116S210005)Q.Wang and J.Wang acknowledge the support from NASA Atmospheric Composition Modeling and Analysis Program(ACMAP,Grant#:80NSSC19K0950).
文摘This paper presents designing sequence-to-sequence recurrent neural network(RNN)architectures for a novel study to predict soil NOx emissions,driven by the imperative of understanding and mitigating environmental impact.The study utilizes data collected by the Environmental Protection Agency(EPA)to develop two distinct RNN predictive models:one built upon the long-short term memory(LSTM)and the other utilizing the gated recurrent unit(GRU).These models are fed with a combination of historical and anticipated air temperature,air moisture,and NOx emissions as inputs to forecast future NOx emissions.Both LSTM and GRU models can capture the intricate pulse patterns inherent in soil NOx emissions.Notably,the GRU model emerges as the superior performer,surpassing the LSTM model in predictive accuracy while demonstrating efficiency by necessitating less training time.Intriguingly,the investigation into varying input features reveals that relying solely on past NOx emissions as input yields satisfactory performance,highlighting the dominant influence of this factor.The study also delves into the impact of altering input series lengths and training data sizes,yielding insights into optimal configurations for enhanced model performance.Importantly,the findings promise to advance our grasp of soil NOx emission dynamics,with implications for environmental management strategies.Looking ahead,the anticipated availability of additional measurements is poised to bolster machine-learning model efficacy.Furthermore,the future study will explore physical-based RNNs,a promising avenue for deeper insights into soil NOx emission prediction.
基金This work was funded by the National Science Foundation of Hunan Province(2020JJ2029)。
文摘Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to the strict time serial dependency,making it difficult to realize a long-term memory function.On the other hand,RNNs cells are highly complex,which will signifi-cantly increase computational complexity and cause waste of computational resources during model training.In this paper,an improved Time Feedforward Connections Recurrent Neural Networks(TFC-RNNs)model was first proposed to address the gradient issue.A parallel branch was introduced for the hidden state at time t−2 to be directly transferred to time t without the nonlinear transforma-tion at time t−1.This is effective in improving the long-term dependence of RNNs.Then,a novel cell structure named Single Gate Recurrent Unit(SGRU)was presented.This cell structure can reduce the number of parameters for RNNs cell,consequently reducing the computational complexity.Next,applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the above two difficulties.Finally,the performance of our proposed TFC-SGRU was verified through sev-eral experiments in terms of long-term memory and anti-interference capabilities.Experimental results demonstrated that our proposed TFC-SGRU model can cap-ture helpful information with time step 1500 and effectively filter out the noise.The TFC-SGRU model accuracy is better than the LSTM and GRU models regarding language processing ability.
基金supported by the BK21 FOUR project funded by the Ministry of Education,Korea(4199990113966).
文摘Lithium-ion batteries are commonly used in electric vehicles,mobile phones,and laptops.These batteries demonstrate several advantages,such as environmental friendliness,high energy density,and long life.However,battery overcharging and overdischarging may occur if the batteries are not monitored continuously.Overcharging causesfire and explosion casualties,and overdischar-ging causes a reduction in the battery capacity and life.In addition,the internal resistance of such batteries varies depending on their external temperature,elec-trolyte,cathode material,and other factors;the capacity of the batteries decreases with temperature.In this study,we develop a method for estimating the state of charge(SOC)using a neural network model that is best suited to the external tem-perature of such batteries based on their characteristics.During our simulation,we acquired data at temperatures of 25°C,30°C,35°C,and 40°C.Based on the tem-perature parameters,the voltage,current,and time parameters were obtained,and six cycles of the parameters based on the temperature were used for the experi-ment.Experimental data to verify the proposed method were obtained through a discharge experiment conducted using a vehicle driving simulator.The experi-mental data were provided as inputs to three types of neural network models:mul-tilayer neural network(MNN),long short-term memory(LSTM),and gated recurrent unit(GRU).The neural network models were trained and optimized for the specific temperatures measured during the experiment,and the SOC was estimated by selecting the most suitable model for each temperature.The experimental results revealed that the mean absolute errors of the MNN,LSTM,and GRU using the proposed method were 2.17%,2.19%,and 2.15%,respec-tively,which are better than those of the conventional method(4.47%,4.60%,and 4.40%).Finally,SOC estimation based on GRU using the proposed method was found to be 2.15%,which was the most accurate.
基金the National Natural Science Foundation of China(Grant Nos.52061135108 and 51976122)the National Science and Technology Major Project(Grant No.2017-III-0007-0033)。
文摘The monitoring of Li-ion battery temperatures is essential to ensure high efficiency and safety.In this work,two types of recurrent neural networks (RNNs),which are long short-term memory-RNN (LSTM-RNN) and gated recurrent unit-RNN(GRU-RNN),are proposed to estimate the surface temperature of 18650 Li-ion batteries during the discharging processes under different ambient temperatures.The datasets acquired from the Prognostics Center of Excellence (PCo E) of NASA are used to train,validate and test the networks.In previous work,temperature has been set as the output of the networks;however,here the temperature difference along the time axis is adopted as the output.The net heat generated results in net temperature change,which is more closely aligned with electrochemical and thermodynamic laws.Extensive simulation results show that the two RNNs can achieve accurate real-time battery temperature estimation.The maximum absolute error in temperature estimation is approximately 0.75°C and the correlation coefficient between the estimated and measured temperature curves is greater than 0.95.The influences of three crucial parameters,which are the number of hidden neurons,initial learning rate and maximum number of iterations,are also assessed in terms of training time,root mean square error and mean absolute error.
基金supported by two Ministry of Education(MoE)Singapore Tier 1 research grants under grant numbers R-296-000-208-133 and R-296-000-241-114.
文摘Computer-empowered detection of possible faults for Heating,Ventilation and Air-Conditioning(HVAC)subsystems,e.g.,chillers,is one of the most important applications in Artificial Intelligence(AI)integrated Internet of Things(IoT).The cyber-physical system greatly enhances the safety and security of the working facilities,reducing time,saving energy and protecting humans’health.Under the current trends of smart building design and energy management optimization,Automated Fault Detection and Diagnosis(AFDD)of chillers integrated with IoT is highly demanded.Recent studies show that standard machine learning techniques,such as Principal Component Analysis(PCA),Support Vector Machine(SVM)and tree-structure-based algorithms,are useful in capturing various chiller faults with high accuracy rates.With the fast development of deep learning technology,Convolutional Neural Networks(CNNs)have been widely and successfully applied to various fields.However,for chiller AFDD,few existing works are adopting CNN and its extensions in the feature extraction and classification processes.In this study,we propose to perform chiller FDD using a CNN-based approach.The proposed approach has two distinct advantages over existing machine learning-based chiller AFDD methods.First,the CNN-based approach does not require the feature selection/extraction process.Since CNN is reputable with its feature extraction capability,the feature extraction and classification processes are merged,leading to a more neat AFDD framework compared to traditional approaches.Second,the classification accuracy is significantly improved compared to traditional methods using the CNN-based approach.
