Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicabilit...Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.展开更多
Landslide displacement prediction can enhance the efficacy of landslide monitoring system,and the prediction of the periodic displacement is particularly challenging.In the previous studies,static regression models(e....Landslide displacement prediction can enhance the efficacy of landslide monitoring system,and the prediction of the periodic displacement is particularly challenging.In the previous studies,static regression models(e.g.,support vector machine(SVM))were mostly used for predicting the periodic displacement.These models may have bad performances,when the dynamic features of landslide triggers are incorporated.This paper proposes a method for predicting the landslide displacement in a dynamic manner,based on the gated recurrent unit(GRU)neural network and complete ensemble empirical decomposition with adaptive noise(CEEMDAN).The CEEMDAN is used to decompose the training data,and the GRU is subsequently used for predicting the periodic displacement.Implementation procedures of the proposed method were illustrated by a case study in the Caojiatuo landslide area,and SVM was also adopted for the periodic displacement prediction.This case study shows that the predictors obtained by SVM are inaccurate,as the landslide displacement is in a pronouncedly step-wise manner.By contrast,the accuracy can be significantly improved using the dynamic predictive method.This paper reveals the significance of capturing the dynamic features of the inputs in the training process,when the machine learning models are adopted to predict the landslide displacement.展开更多
Based on a comparative analysis of the Long Short-Term Memory(LSTM)and Gated Recurrent Unit(GRU)networks,we optimize the structure of the GRU network and propose a new modulation recognition method based on feature ex...Based on a comparative analysis of the Long Short-Term Memory(LSTM)and Gated Recurrent Unit(GRU)networks,we optimize the structure of the GRU network and propose a new modulation recognition method based on feature extraction and a deep learning algorithm.High-order cumulant,Signal-to-Noise Ratio(SNR),instantaneous feature,and the cyclic spectrum of signals are extracted firstly,and then input into the Convolutional Neural Network(CNN)and the parallel network of GRU for recognition.Eight modulation modes of communication signals are recognized automatically.Simulation results show that the proposed method can achieve high recognition rate at low SNR.展开更多
Static Poisson’s ratio(vs)is crucial for determining geomechanical properties in petroleum applications,namely sand production.Some models have been used to predict vs;however,the published models were limited to spe...Static Poisson’s ratio(vs)is crucial for determining geomechanical properties in petroleum applications,namely sand production.Some models have been used to predict vs;however,the published models were limited to specific data ranges with an average absolute percentage relative error(AAPRE)of more than 10%.The published gated recurrent unit(GRU)models do not consider trend analysis to show physical behaviors.In this study,we aim to develop a GRU model using trend analysis and three inputs for predicting n s based on a broad range of data,n s(value of 0.1627-0.4492),bulk formation density(RHOB)(0.315-2.994 g/mL),compressional time(DTc)(44.43-186.9 μs/ft),and shear time(DTs)(72.9-341.2μ s/ft).The GRU model was evaluated using different approaches,including statistical error an-alyses.The GRU model showed the proper trends,and the model data ranges were wider than previous ones.The GRU model has the largest correlation coefficient(R)of 0.967 and the lowest AAPRE,average percent relative error(APRE),root mean square error(RMSE),and standard deviation(SD)of 3.228%,1.054%,4.389,and 0.013,respectively,compared to other models.The GRU model has a high accuracy for the different datasets:training,validation,testing,and the whole datasets with R and AAPRE values were 0.981 and 2.601%,0.966 and 3.274%,0.967 and 3.228%,and 0.977 and 2.861%,respectively.The group error analyses of all inputs show that the GRU model has less than 5% AAPRE for all input ranges,which is superior to other models that have different AAPRE values of more than 10% at various ranges of inputs.展开更多
Accurate prediction of the state-of-charge(SOC)of battery energy storage system(BESS)is critical for its safety and lifespan in electric vehicles.To overcome the imbalance of existing methods between multi-scale featu...Accurate prediction of the state-of-charge(SOC)of battery energy storage system(BESS)is critical for its safety and lifespan in electric vehicles.To overcome the imbalance of existing methods between multi-scale feature fusion and global feature extraction,this paper introduces a novel multi-scale fusion(MSF)model based on gated recurrent unit(GRU),which is specifically designed for complex multi-step SOC prediction in practical BESSs.Pearson correlation analysis is first employed to identify SOC-related parameters.These parameters are then input into a multi-layer GRU for point-wise feature extraction.Concurrently,the parameters undergo patching before entering a dual-stage multi-layer GRU,thus enabling the model to capture nuanced information across varying time intervals.Ultimately,by means of adaptive weight fusion and a fully connected network,multi-step SOC predictions are rendered.Following extensive validation over multiple days,it is illustrated that the proposed model achieves an absolute error of less than 1.5%in real-time SOC prediction.展开更多
The rapid growth of the Chinese economy has fueled the expansion of power grids.Power transformers are key equipment in power grid projects,and their price changes have a significant impact on cost control.However,the...The rapid growth of the Chinese economy has fueled the expansion of power grids.Power transformers are key equipment in power grid projects,and their price changes have a significant impact on cost control.However,the prices of power transformer materials manifest as nonsmooth and nonlinear sequences.Hence,estimating the acquisition costs of power grid projects is difficult,hindering the normal operation of power engineering construction.To more accurately predict the price of power transformer materials,this study proposes a method based on complementary ensemble empirical mode decomposition(CEEMD)and gated recurrent unit(GRU)network.First,the CEEMD decomposed the price series into multiple intrinsic mode functions(IMFs).Multiple IMFs were clustered to obtain several aggregated sequences based on the sample entropy of each IMF.Then,an empirical wavelet transform(EWT)was applied to the aggregation sequence with a large sample entropy,and the multiple subsequences obtained from the decomposition were predicted by the GRU model.The GRU model was used to directly predict the aggregation sequences with a small sample entropy.In this study,we used authentic historical pricing data for power transformer materials to validate the proposed approach.The empirical findings demonstrated the efficacy of our method across both datasets,with mean absolute percentage errors(MAPEs)of less than 1%and 3%.This approach holds a significant reference value for future research in the field of power transformer material price prediction.展开更多
Short-term load forecasting(STLF)plays a crucial role in the smart grid.However,it is challenging to capture the long-time dependence and the nonlinear relationship due to the comprehensive fluctuations of the electri...Short-term load forecasting(STLF)plays a crucial role in the smart grid.However,it is challenging to capture the long-time dependence and the nonlinear relationship due to the comprehensive fluctuations of the electrical load.In this paper,an STLF model based on gated recurrent unit and multi-head attention(GRU-MA)is proposed to address the aforementioned problems.The proposed model accommodates the time series and nonlinear relationship of load data through gated recurrent unit(GRU)and exploits multi-head attention(MA)to learn the decisive features and long-term dependencies.Additionally,the proposed model is compared with the support vector regression(SVR)model,the recurrent neural network and multi-head attention(RNN-MA)model,the long short-term memory and multi-head attention(LSTM-MA)model,the GRU model,and the temporal convolutional network(TCN)model using the public dataset of the Global Energy Forecasting Competition 2014(GEFCOM2014).The results demonstrate that the GRU-MA model has the best prediction accuracy.展开更多
The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). I...The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). In this study, we propose a new method, GRU_Z-ET, by introducing Z and ET as two independent variables into the GRU neural network to conduct the quantitative single-polarization radar precipitation estimation. The performance of GRU_Z-ET is compared with that of the other three methods in three heavy rainfall cases in China during 2018, namely, the traditional Z-R relationship(Z=300R1.4), the optimal Z-R relationship(Z=79R1.68) and the GRU neural network with only Z as the independent input variable(GRU_Z). The results indicate that the GRU_Z-ET performs the best, while the traditional Z-R relationship performs the worst. The performances of the rest two methods are similar.To further evaluate the performance of the GRU_Z-ET, 200 rainfall events with 21882 total samples during May–July of 2018 are used for statistical analysis. Results demonstrate that the spatial correlation coefficients, threat scores and probability of detection between the observed and estimated precipitation are the largest for the GRU_Z-ET and the smallest for the traditional Z-R relationship, and the root mean square error is just the opposite. In addition, these statistics of GRU_Z are similar to those of optimal Z-R relationship. Thus, it can be concluded that the performance of the GRU_ZET is the best in the four methods for the quantitative precipitation estimation.展开更多
This study proposed a new real-time manufacturing process monitoring method to monitor and detect process shifts in manufacturing operations.Since real-time production process monitoring is critical in today’s smart ...This study proposed a new real-time manufacturing process monitoring method to monitor and detect process shifts in manufacturing operations.Since real-time production process monitoring is critical in today’s smart manufacturing.The more robust the monitoring model,the more reliable a process is to be under control.In the past,many researchers have developed real-time monitoring methods to detect process shifts early.However,thesemethods have limitations in detecting process shifts as quickly as possible and handling various data volumes and varieties.In this paper,a robust monitoring model combining Gated Recurrent Unit(GRU)and Random Forest(RF)with Real-Time Contrast(RTC)called GRU-RF-RTC was proposed to detect process shifts rapidly.The effectiveness of the proposed GRU-RF-RTC model is first evaluated using multivariate normal and nonnormal distribution datasets.Then,to prove the applicability of the proposed model in a realmanufacturing setting,the model was evaluated using real-world normal and non-normal problems.The results demonstrate that the proposed GRU-RF-RTC outperforms other methods in detecting process shifts quickly with the lowest average out-of-control run length(ARL1)in all synthesis and real-world problems under normal and non-normal cases.The experiment results on real-world problems highlight the significance of the proposed GRU-RF-RTC model in modern manufacturing process monitoring applications.The result reveals that the proposed method improves the shift detection capability by 42.14%in normal and 43.64%in gamma distribution problems.展开更多
To predict renewable energy sources such as solar power in microgrids more accurately,a hybrid power prediction method is presented in this paper.First,the self-attention mechanism is introduced based on a bidirection...To predict renewable energy sources such as solar power in microgrids more accurately,a hybrid power prediction method is presented in this paper.First,the self-attention mechanism is introduced based on a bidirectional gated recurrent neural network(BiGRU)to explore the time-series characteristics of solar power output and consider the influence of different time nodes on the prediction results.Subsequently,an improved quantum particle swarm optimization(QPSO)algorithm is proposed to optimize the hyperparameters of the combined prediction model.The final proposed LQPSO-BiGRU-self-attention hybrid model can predict solar power more effectively.In addition,considering the coordinated utilization of various energy sources such as electricity,hydrogen,and renewable energy,a multi-objective optimization model that considers both economic and environmental costs was constructed.A two-stage adaptive multi-objective quantum particle swarm optimization algorithm aided by a Lévy flight,named MO-LQPSO,was proposed for the comprehensive optimal scheduling of a multi-energy microgrid system.This algorithm effectively balances the global and local search capabilities and enhances the solution of complex nonlinear problems.The effectiveness and superiority of the proposed scheme are verified through comparative simulations.展开更多
As the global demand for renewable energy grows,solar energy is gaining attention as a clean,sustainable energy source.Accurate assessment of solar energy resources is crucial for the siting and design of photovoltaic...As the global demand for renewable energy grows,solar energy is gaining attention as a clean,sustainable energy source.Accurate assessment of solar energy resources is crucial for the siting and design of photovoltaic power plants.This study proposes an integrated deep learning-based photovoltaic resource assessment method.Ensemble learning and deep learning methods are fused for photovoltaic resource assessment for the first time.The proposed method combines the random forest,gated recurrent unit,and long short-term memory to effectively improve the accuracy and reliability of photovoltaic resource assessment.The proposed method has strong adaptability and high accuracy even in the photovoltaic resource assessment of complex terrain and landscape.The experimental results show that the proposed method outperforms the comparison algorithm in all evaluation indexes,indicating that the proposed method has higher accuracy and reliability in photovoltaic resource assessment with improved generalization performance traditional single algorithm.展开更多
While encryption technology safeguards the security of network communications,malicious traffic also uses encryption protocols to obscure its malicious behavior.To address the issues of traditional machine learning me...While encryption technology safeguards the security of network communications,malicious traffic also uses encryption protocols to obscure its malicious behavior.To address the issues of traditional machine learning methods relying on expert experience and the insufficient representation capabilities of existing deep learning methods for encrypted malicious traffic,we propose an encrypted malicious traffic classification method that integrates global semantic features with local spatiotemporal features,called BERT-based Spatio-Temporal Features Network(BSTFNet).At the packet-level granularity,the model captures the global semantic features of packets through the attention mechanism of the Bidirectional Encoder Representations from Transformers(BERT)model.At the byte-level granularity,we initially employ the Bidirectional Gated Recurrent Unit(BiGRU)model to extract temporal features from bytes,followed by the utilization of the Text Convolutional Neural Network(TextCNN)model with multi-sized convolution kernels to extract local multi-receptive field spatial features.The fusion of features from both granularities serves as the ultimate multidimensional representation of malicious traffic.Our approach achieves accuracy and F1-score of 99.39%and 99.40%,respectively,on the publicly available USTC-TFC2016 dataset,and effectively reduces sample confusion within the Neris and Virut categories.The experimental results demonstrate that our method has outstanding representation and classification capabilities for encrypted malicious traffic.展开更多
Considering the nonlinear structure and spatial-temporal correlation of traffic network,and the influence of potential correlation between nodes of traffic network on the spatial features,this paper proposes a traffic...Considering the nonlinear structure and spatial-temporal correlation of traffic network,and the influence of potential correlation between nodes of traffic network on the spatial features,this paper proposes a traffic speed prediction model based on the combination of graph attention network with self-adaptive adjacency matrix(SAdpGAT)and bidirectional gated recurrent unit(BiGRU).First-ly,the model introduces graph attention network(GAT)to extract the spatial features of real road network and potential road network respectively in spatial dimension.Secondly,the spatial features are input into BiGRU to extract the time series features.Finally,the prediction results of the real road network and the potential road network are connected to generate the final prediction results of the model.The experimental results show that the prediction accuracy of the proposed model is im-proved obviously on METR-LA and PEMS-BAY datasets,which proves the advantages of the pro-posed spatial-temporal model in traffic speed prediction.展开更多
Aerial threat assessment is a crucial link in modern air combat, whose result counts a great deal for commanders to make decisions. With the consideration that the existing threat assessment methods have difficulties ...Aerial threat assessment is a crucial link in modern air combat, whose result counts a great deal for commanders to make decisions. With the consideration that the existing threat assessment methods have difficulties in dealing with high dimensional time series target data, a threat assessment method based on self-attention mechanism and gated recurrent unit(SAGRU) is proposed. Firstly, a threat feature system including air combat situations and capability features is established. Moreover, a data augmentation process based on fractional Fourier transform(FRFT) is applied to extract more valuable information from time series situation features. Furthermore, aiming to capture key characteristics of battlefield evolution, a bidirectional GRU and SA mechanisms are designed for enhanced features.Subsequently, after the concatenation of the processed air combat situation and capability features, the target threat level will be predicted by fully connected neural layers and the softmax classifier. Finally, in order to validate this model, an air combat dataset generated by a combat simulation system is introduced for model training and testing. The comparison experiments show the proposed model has structural rationality and can perform threat assessment faster and more accurately than the other existing models based on deep learning.展开更多
现有的方面级情感分析方法对句法依存树蕴含信息使用不足,忽略多方面词之间的关联,并且缺少对外部知识的使用。针对这些问题,提出一种知识增强的方面词交互图神经网络(KEAIG)模型。首先利用融合领域知识的BERT-PT (Bidirectional Encode...现有的方面级情感分析方法对句法依存树蕴含信息使用不足,忽略多方面词之间的关联,并且缺少对外部知识的使用。针对这些问题,提出一种知识增强的方面词交互图神经网络(KEAIG)模型。首先利用融合领域知识的BERT-PT (Bidirectional Encoder Representation from Transformers with Post-Train)编码文本,并利用知识图谱增加句法树的情感信息。模型分两部分对句法依存树蕴含的信息进行提取:第一部分利用句法依存树中的关联关系和每个单词的词性标签提取句子特征,第二部分对融入知识图谱的句法依存树进行特征提取。之后使用融合门控单元将多方面词关联特征融合进提取到的特征中。最后将两部分句子表示拼接起来作为最终分类依据。在4个数据集上的实验结果表明,所提模型相较于基准模型关系图注意力网络(RGAT),在准确率上分别提升了2.17%、5.54%、2.60%和2.83%,在F1值(Macro-F1)上分别提升了2.69%、6.87%、8.77%和14.70%,充分表明了利用句法树、引入外部知识和提取多方面词关联的有效性。展开更多
In order to improve the performance degradation prediction accuracy of proton exchange membrane fuel cell(PEMFC),a fusion prediction method(CKDG)based on adaptive noise complete ensemble empirical mode decomposition(C...In order to improve the performance degradation prediction accuracy of proton exchange membrane fuel cell(PEMFC),a fusion prediction method(CKDG)based on adaptive noise complete ensemble empirical mode decomposition(CEEMDAN),kernel principal component analysis(KPCA)and dual attention mechanism gated recurrent unit neural network(DA-GRU)was proposed.CEEMDAN and KPCA were used to extract the input feature data sequence,reduce the influence of random factors,and capture essential feature components to reduce the model complexity.The DA-GRU network helps to learn the feature mapping relationship of data in long time series and predict the changing trend of performance degradation data more accurately.The actual aging experimental data verify the performance of the CKDG method.The results show that under the steady-state condition of 20%training data prediction,the CKDA method can reduce the root mean square error(RMSE)by 52.7%and 34.6%,respectively,compared with the traditional LSTM and GRU neural networks.Compared with the simple DA-GRU network,RMSE is reduced by 15%,and the degree of over-fitting is reduced,which has higher accuracy.It also shows excellent prediction performance under the dynamic condition data set and has good universality.展开更多
Emotion recognition from speech data is an active and emerging area of research that plays an important role in numerous applications,such as robotics,virtual reality,behavior assessments,and emergency call centers.Re...Emotion recognition from speech data is an active and emerging area of research that plays an important role in numerous applications,such as robotics,virtual reality,behavior assessments,and emergency call centers.Recently,researchers have developed many techniques in this field in order to ensure an improvement in the accuracy by utilizing several deep learning approaches,but the recognition rate is still not convincing.Our main aim is to develop a new technique that increases the recognition rate with reasonable cost computations.In this paper,we suggested a new technique,which is a one-dimensional dilated convolutional neural network(1D-DCNN)for speech emotion recognition(SER)that utilizes the hierarchical features learning blocks(HFLBs)with a bi-directional gated recurrent unit(BiGRU).We designed a one-dimensional CNN network to enhance the speech signals,which uses a spectral analysis,and to extract the hidden patterns from the speech signals that are fed into a stacked one-dimensional dilated network that are called HFLBs.Each HFLB contains one dilated convolution layer(DCL),one batch normalization(BN),and one leaky_relu(Relu)layer in order to extract the emotional features using a hieratical correlation strategy.Furthermore,the learned emotional features are feed into a BiGRU in order to adjust the global weights and to recognize the temporal cues.The final state of the deep BiGRU is passed from a softmax classifier in order to produce the probabilities of the emotions.The proposed model was evaluated over three benchmarked datasets that included the IEMOCAP,EMO-DB,and RAVDESS,which achieved 72.75%,91.14%,and 78.01%accuracy,respectively.展开更多
Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularl...Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.展开更多
基金supported by the Natural Science Foundation of China(Grant Nos.51979158,51639008,51679135,and 51422905)the Program of Shanghai Academic Research Leader by Science and Technology Commission of Shanghai Municipality(Project No.19XD1421900)。
文摘Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.
基金The authors appreciate the financial support provided by the Natural Science Foundation of China(No.41807294)This study was also financially supported by China Geological Survey Project(Nos.DD20190716 and 0001212020CC60002)。
文摘Landslide displacement prediction can enhance the efficacy of landslide monitoring system,and the prediction of the periodic displacement is particularly challenging.In the previous studies,static regression models(e.g.,support vector machine(SVM))were mostly used for predicting the periodic displacement.These models may have bad performances,when the dynamic features of landslide triggers are incorporated.This paper proposes a method for predicting the landslide displacement in a dynamic manner,based on the gated recurrent unit(GRU)neural network and complete ensemble empirical decomposition with adaptive noise(CEEMDAN).The CEEMDAN is used to decompose the training data,and the GRU is subsequently used for predicting the periodic displacement.Implementation procedures of the proposed method were illustrated by a case study in the Caojiatuo landslide area,and SVM was also adopted for the periodic displacement prediction.This case study shows that the predictors obtained by SVM are inaccurate,as the landslide displacement is in a pronouncedly step-wise manner.By contrast,the accuracy can be significantly improved using the dynamic predictive method.This paper reveals the significance of capturing the dynamic features of the inputs in the training process,when the machine learning models are adopted to predict the landslide displacement.
基金partially supported by Major Scientific and Technological Achievements Transformation Project of Heilongjiang Province in 2019(No.CG20A007)。
文摘Based on a comparative analysis of the Long Short-Term Memory(LSTM)and Gated Recurrent Unit(GRU)networks,we optimize the structure of the GRU network and propose a new modulation recognition method based on feature extraction and a deep learning algorithm.High-order cumulant,Signal-to-Noise Ratio(SNR),instantaneous feature,and the cyclic spectrum of signals are extracted firstly,and then input into the Convolutional Neural Network(CNN)and the parallel network of GRU for recognition.Eight modulation modes of communication signals are recognized automatically.Simulation results show that the proposed method can achieve high recognition rate at low SNR.
基金The authors thank the Yayasan Universiti Teknologi PETRONAS(YUTP FRG Grant No.015LC0-428)at Universiti Teknologi PETRO-NAS for supporting this study.
文摘Static Poisson’s ratio(vs)is crucial for determining geomechanical properties in petroleum applications,namely sand production.Some models have been used to predict vs;however,the published models were limited to specific data ranges with an average absolute percentage relative error(AAPRE)of more than 10%.The published gated recurrent unit(GRU)models do not consider trend analysis to show physical behaviors.In this study,we aim to develop a GRU model using trend analysis and three inputs for predicting n s based on a broad range of data,n s(value of 0.1627-0.4492),bulk formation density(RHOB)(0.315-2.994 g/mL),compressional time(DTc)(44.43-186.9 μs/ft),and shear time(DTs)(72.9-341.2μ s/ft).The GRU model was evaluated using different approaches,including statistical error an-alyses.The GRU model showed the proper trends,and the model data ranges were wider than previous ones.The GRU model has the largest correlation coefficient(R)of 0.967 and the lowest AAPRE,average percent relative error(APRE),root mean square error(RMSE),and standard deviation(SD)of 3.228%,1.054%,4.389,and 0.013,respectively,compared to other models.The GRU model has a high accuracy for the different datasets:training,validation,testing,and the whole datasets with R and AAPRE values were 0.981 and 2.601%,0.966 and 3.274%,0.967 and 3.228%,and 0.977 and 2.861%,respectively.The group error analyses of all inputs show that the GRU model has less than 5% AAPRE for all input ranges,which is superior to other models that have different AAPRE values of more than 10% at various ranges of inputs.
基金supported in part by the National Natural Science Foundation of China(No.62172036).
文摘Accurate prediction of the state-of-charge(SOC)of battery energy storage system(BESS)is critical for its safety and lifespan in electric vehicles.To overcome the imbalance of existing methods between multi-scale feature fusion and global feature extraction,this paper introduces a novel multi-scale fusion(MSF)model based on gated recurrent unit(GRU),which is specifically designed for complex multi-step SOC prediction in practical BESSs.Pearson correlation analysis is first employed to identify SOC-related parameters.These parameters are then input into a multi-layer GRU for point-wise feature extraction.Concurrently,the parameters undergo patching before entering a dual-stage multi-layer GRU,thus enabling the model to capture nuanced information across varying time intervals.Ultimately,by means of adaptive weight fusion and a fully connected network,multi-step SOC predictions are rendered.Following extensive validation over multiple days,it is illustrated that the proposed model achieves an absolute error of less than 1.5%in real-time SOC prediction.
基金supported by China Southern Power Grid Science and Technology Innovation Research Project(000000KK52220052).
文摘The rapid growth of the Chinese economy has fueled the expansion of power grids.Power transformers are key equipment in power grid projects,and their price changes have a significant impact on cost control.However,the prices of power transformer materials manifest as nonsmooth and nonlinear sequences.Hence,estimating the acquisition costs of power grid projects is difficult,hindering the normal operation of power engineering construction.To more accurately predict the price of power transformer materials,this study proposes a method based on complementary ensemble empirical mode decomposition(CEEMD)and gated recurrent unit(GRU)network.First,the CEEMD decomposed the price series into multiple intrinsic mode functions(IMFs).Multiple IMFs were clustered to obtain several aggregated sequences based on the sample entropy of each IMF.Then,an empirical wavelet transform(EWT)was applied to the aggregation sequence with a large sample entropy,and the multiple subsequences obtained from the decomposition were predicted by the GRU model.The GRU model was used to directly predict the aggregation sequences with a small sample entropy.In this study,we used authentic historical pricing data for power transformer materials to validate the proposed approach.The empirical findings demonstrated the efficacy of our method across both datasets,with mean absolute percentage errors(MAPEs)of less than 1%and 3%.This approach holds a significant reference value for future research in the field of power transformer material price prediction.
基金supported by the National Natural Science Foundation of China(61771258)。
文摘Short-term load forecasting(STLF)plays a crucial role in the smart grid.However,it is challenging to capture the long-time dependence and the nonlinear relationship due to the comprehensive fluctuations of the electrical load.In this paper,an STLF model based on gated recurrent unit and multi-head attention(GRU-MA)is proposed to address the aforementioned problems.The proposed model accommodates the time series and nonlinear relationship of load data through gated recurrent unit(GRU)and exploits multi-head attention(MA)to learn the decisive features and long-term dependencies.Additionally,the proposed model is compared with the support vector regression(SVR)model,the recurrent neural network and multi-head attention(RNN-MA)model,the long short-term memory and multi-head attention(LSTM-MA)model,the GRU model,and the temporal convolutional network(TCN)model using the public dataset of the Global Energy Forecasting Competition 2014(GEFCOM2014).The results demonstrate that the GRU-MA model has the best prediction accuracy.
基金jointly supported by the National Science Foundation of China (Grant Nos. 42275007 and 41865003)Jiangxi Provincial Department of science and technology project (Grant No. 20171BBG70004)。
文摘The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). In this study, we propose a new method, GRU_Z-ET, by introducing Z and ET as two independent variables into the GRU neural network to conduct the quantitative single-polarization radar precipitation estimation. The performance of GRU_Z-ET is compared with that of the other three methods in three heavy rainfall cases in China during 2018, namely, the traditional Z-R relationship(Z=300R1.4), the optimal Z-R relationship(Z=79R1.68) and the GRU neural network with only Z as the independent input variable(GRU_Z). The results indicate that the GRU_Z-ET performs the best, while the traditional Z-R relationship performs the worst. The performances of the rest two methods are similar.To further evaluate the performance of the GRU_Z-ET, 200 rainfall events with 21882 total samples during May–July of 2018 are used for statistical analysis. Results demonstrate that the spatial correlation coefficients, threat scores and probability of detection between the observed and estimated precipitation are the largest for the GRU_Z-ET and the smallest for the traditional Z-R relationship, and the root mean square error is just the opposite. In addition, these statistics of GRU_Z are similar to those of optimal Z-R relationship. Thus, it can be concluded that the performance of the GRU_ZET is the best in the four methods for the quantitative precipitation estimation.
基金support from the National Science and Technology Council of Taiwan(Contract Nos.111-2221 E-011081 and 111-2622-E-011019)the support from Intelligent Manufacturing Innovation Center(IMIC),National Taiwan University of Science and Technology(NTUST),Taipei,Taiwan,which is a Featured Areas Research Center in Higher Education Sprout Project of Ministry of Education(MOE),Taiwan(since 2023)was appreciatedWe also thank Wang Jhan Yang Charitable Trust Fund(Contract No.WJY 2020-HR-01)for its financial support.
文摘This study proposed a new real-time manufacturing process monitoring method to monitor and detect process shifts in manufacturing operations.Since real-time production process monitoring is critical in today’s smart manufacturing.The more robust the monitoring model,the more reliable a process is to be under control.In the past,many researchers have developed real-time monitoring methods to detect process shifts early.However,thesemethods have limitations in detecting process shifts as quickly as possible and handling various data volumes and varieties.In this paper,a robust monitoring model combining Gated Recurrent Unit(GRU)and Random Forest(RF)with Real-Time Contrast(RTC)called GRU-RF-RTC was proposed to detect process shifts rapidly.The effectiveness of the proposed GRU-RF-RTC model is first evaluated using multivariate normal and nonnormal distribution datasets.Then,to prove the applicability of the proposed model in a realmanufacturing setting,the model was evaluated using real-world normal and non-normal problems.The results demonstrate that the proposed GRU-RF-RTC outperforms other methods in detecting process shifts quickly with the lowest average out-of-control run length(ARL1)in all synthesis and real-world problems under normal and non-normal cases.The experiment results on real-world problems highlight the significance of the proposed GRU-RF-RTC model in modern manufacturing process monitoring applications.The result reveals that the proposed method improves the shift detection capability by 42.14%in normal and 43.64%in gamma distribution problems.
基金supported by the National Natural Science Foundation of China under Grant 51977004the Beijing Natural Science Foundation under Grant 4212042.
文摘To predict renewable energy sources such as solar power in microgrids more accurately,a hybrid power prediction method is presented in this paper.First,the self-attention mechanism is introduced based on a bidirectional gated recurrent neural network(BiGRU)to explore the time-series characteristics of solar power output and consider the influence of different time nodes on the prediction results.Subsequently,an improved quantum particle swarm optimization(QPSO)algorithm is proposed to optimize the hyperparameters of the combined prediction model.The final proposed LQPSO-BiGRU-self-attention hybrid model can predict solar power more effectively.In addition,considering the coordinated utilization of various energy sources such as electricity,hydrogen,and renewable energy,a multi-objective optimization model that considers both economic and environmental costs was constructed.A two-stage adaptive multi-objective quantum particle swarm optimization algorithm aided by a Lévy flight,named MO-LQPSO,was proposed for the comprehensive optimal scheduling of a multi-energy microgrid system.This algorithm effectively balances the global and local search capabilities and enhances the solution of complex nonlinear problems.The effectiveness and superiority of the proposed scheme are verified through comparative simulations.
基金funded by Key-Area Research and Development Program Project of Guangdong Province (2021B0101230003)China Southern Power Grid Science and Technology Project (ZBKJXM20220004).
文摘As the global demand for renewable energy grows,solar energy is gaining attention as a clean,sustainable energy source.Accurate assessment of solar energy resources is crucial for the siting and design of photovoltaic power plants.This study proposes an integrated deep learning-based photovoltaic resource assessment method.Ensemble learning and deep learning methods are fused for photovoltaic resource assessment for the first time.The proposed method combines the random forest,gated recurrent unit,and long short-term memory to effectively improve the accuracy and reliability of photovoltaic resource assessment.The proposed method has strong adaptability and high accuracy even in the photovoltaic resource assessment of complex terrain and landscape.The experimental results show that the proposed method outperforms the comparison algorithm in all evaluation indexes,indicating that the proposed method has higher accuracy and reliability in photovoltaic resource assessment with improved generalization performance traditional single algorithm.
基金This research was funded by National Natural Science Foundation of China under Grant No.61806171Sichuan University of Science&Engineering Talent Project under Grant No.2021RC15+2 种基金Open Fund Project of Key Laboratory for Non-Destructive Testing and Engineering Computer of Sichuan Province Universities on Bridge Inspection and Engineering under Grant No.2022QYJ06Sichuan University of Science&Engineering Graduate Student Innovation Fund under Grant No.Y2023115The Scientific Research and Innovation Team Program of Sichuan University of Science and Technology under Grant No.SUSE652A006.
文摘While encryption technology safeguards the security of network communications,malicious traffic also uses encryption protocols to obscure its malicious behavior.To address the issues of traditional machine learning methods relying on expert experience and the insufficient representation capabilities of existing deep learning methods for encrypted malicious traffic,we propose an encrypted malicious traffic classification method that integrates global semantic features with local spatiotemporal features,called BERT-based Spatio-Temporal Features Network(BSTFNet).At the packet-level granularity,the model captures the global semantic features of packets through the attention mechanism of the Bidirectional Encoder Representations from Transformers(BERT)model.At the byte-level granularity,we initially employ the Bidirectional Gated Recurrent Unit(BiGRU)model to extract temporal features from bytes,followed by the utilization of the Text Convolutional Neural Network(TextCNN)model with multi-sized convolution kernels to extract local multi-receptive field spatial features.The fusion of features from both granularities serves as the ultimate multidimensional representation of malicious traffic.Our approach achieves accuracy and F1-score of 99.39%and 99.40%,respectively,on the publicly available USTC-TFC2016 dataset,and effectively reduces sample confusion within the Neris and Virut categories.The experimental results demonstrate that our method has outstanding representation and classification capabilities for encrypted malicious traffic.
基金the National Natural Science Foundation of China(No.61461027,61762059)the Provincial Science and Technology Program supported the Key Project of Natural Science Foundation of Gansu Province(No.22JR5RA226)。
文摘Considering the nonlinear structure and spatial-temporal correlation of traffic network,and the influence of potential correlation between nodes of traffic network on the spatial features,this paper proposes a traffic speed prediction model based on the combination of graph attention network with self-adaptive adjacency matrix(SAdpGAT)and bidirectional gated recurrent unit(BiGRU).First-ly,the model introduces graph attention network(GAT)to extract the spatial features of real road network and potential road network respectively in spatial dimension.Secondly,the spatial features are input into BiGRU to extract the time series features.Finally,the prediction results of the real road network and the potential road network are connected to generate the final prediction results of the model.The experimental results show that the prediction accuracy of the proposed model is im-proved obviously on METR-LA and PEMS-BAY datasets,which proves the advantages of the pro-posed spatial-temporal model in traffic speed prediction.
基金supported by the National Natural Science Foundation of China (6202201562088101)+1 种基金Shanghai Municipal Science and Technology Major Project (2021SHZDZX0100)Shanghai Municip al Commission of Science and Technology Project (19511132101)。
文摘Aerial threat assessment is a crucial link in modern air combat, whose result counts a great deal for commanders to make decisions. With the consideration that the existing threat assessment methods have difficulties in dealing with high dimensional time series target data, a threat assessment method based on self-attention mechanism and gated recurrent unit(SAGRU) is proposed. Firstly, a threat feature system including air combat situations and capability features is established. Moreover, a data augmentation process based on fractional Fourier transform(FRFT) is applied to extract more valuable information from time series situation features. Furthermore, aiming to capture key characteristics of battlefield evolution, a bidirectional GRU and SA mechanisms are designed for enhanced features.Subsequently, after the concatenation of the processed air combat situation and capability features, the target threat level will be predicted by fully connected neural layers and the softmax classifier. Finally, in order to validate this model, an air combat dataset generated by a combat simulation system is introduced for model training and testing. The comparison experiments show the proposed model has structural rationality and can perform threat assessment faster and more accurately than the other existing models based on deep learning.
文摘现有的方面级情感分析方法对句法依存树蕴含信息使用不足,忽略多方面词之间的关联,并且缺少对外部知识的使用。针对这些问题,提出一种知识增强的方面词交互图神经网络(KEAIG)模型。首先利用融合领域知识的BERT-PT (Bidirectional Encoder Representation from Transformers with Post-Train)编码文本,并利用知识图谱增加句法树的情感信息。模型分两部分对句法依存树蕴含的信息进行提取:第一部分利用句法依存树中的关联关系和每个单词的词性标签提取句子特征,第二部分对融入知识图谱的句法依存树进行特征提取。之后使用融合门控单元将多方面词关联特征融合进提取到的特征中。最后将两部分句子表示拼接起来作为最终分类依据。在4个数据集上的实验结果表明,所提模型相较于基准模型关系图注意力网络(RGAT),在准确率上分别提升了2.17%、5.54%、2.60%和2.83%,在F1值(Macro-F1)上分别提升了2.69%、6.87%、8.77%和14.70%,充分表明了利用句法树、引入外部知识和提取多方面词关联的有效性。
基金funded by Shaanxi Province Key Industrial Chain Project(2023-ZDLGY-24)Industrialization Project of Shaanxi Provincial Education Department(21JC018)+1 种基金Shaanxi Province Key Research and Development Program(2021ZDLGY13-02)the Open Foundation of State Key Laboratory for Advanced Metals and Materials(2022-Z01).
文摘In order to improve the performance degradation prediction accuracy of proton exchange membrane fuel cell(PEMFC),a fusion prediction method(CKDG)based on adaptive noise complete ensemble empirical mode decomposition(CEEMDAN),kernel principal component analysis(KPCA)and dual attention mechanism gated recurrent unit neural network(DA-GRU)was proposed.CEEMDAN and KPCA were used to extract the input feature data sequence,reduce the influence of random factors,and capture essential feature components to reduce the model complexity.The DA-GRU network helps to learn the feature mapping relationship of data in long time series and predict the changing trend of performance degradation data more accurately.The actual aging experimental data verify the performance of the CKDG method.The results show that under the steady-state condition of 20%training data prediction,the CKDA method can reduce the root mean square error(RMSE)by 52.7%and 34.6%,respectively,compared with the traditional LSTM and GRU neural networks.Compared with the simple DA-GRU network,RMSE is reduced by 15%,and the degree of over-fitting is reduced,which has higher accuracy.It also shows excellent prediction performance under the dynamic condition data set and has good universality.
基金supported by the National Research Foundation of Korea funded by the Korean Government through the Ministry of Science and ICT under Grant NRF-2020R1F1A1060659 and in part by the 2020 Faculty Research Fund of Sejong University。
文摘Emotion recognition from speech data is an active and emerging area of research that plays an important role in numerous applications,such as robotics,virtual reality,behavior assessments,and emergency call centers.Recently,researchers have developed many techniques in this field in order to ensure an improvement in the accuracy by utilizing several deep learning approaches,but the recognition rate is still not convincing.Our main aim is to develop a new technique that increases the recognition rate with reasonable cost computations.In this paper,we suggested a new technique,which is a one-dimensional dilated convolutional neural network(1D-DCNN)for speech emotion recognition(SER)that utilizes the hierarchical features learning blocks(HFLBs)with a bi-directional gated recurrent unit(BiGRU).We designed a one-dimensional CNN network to enhance the speech signals,which uses a spectral analysis,and to extract the hidden patterns from the speech signals that are fed into a stacked one-dimensional dilated network that are called HFLBs.Each HFLB contains one dilated convolution layer(DCL),one batch normalization(BN),and one leaky_relu(Relu)layer in order to extract the emotional features using a hieratical correlation strategy.Furthermore,the learned emotional features are feed into a BiGRU in order to adjust the global weights and to recognize the temporal cues.The final state of the deep BiGRU is passed from a softmax classifier in order to produce the probabilities of the emotions.The proposed model was evaluated over three benchmarked datasets that included the IEMOCAP,EMO-DB,and RAVDESS,which achieved 72.75%,91.14%,and 78.01%accuracy,respectively.
文摘Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.
