针对传统滑坡位移预测模型存在对历史数据遗忘的问题,提出了一种基于长短时记忆(long short time memory,LSTM)网络的滑坡位移动态预测模型。首先,将滑坡累计位移分解为趋势项位移与波动项位移,利用多项式拟合预测趋势项位移;然后,通过...针对传统滑坡位移预测模型存在对历史数据遗忘的问题,提出了一种基于长短时记忆(long short time memory,LSTM)网络的滑坡位移动态预测模型。首先,将滑坡累计位移分解为趋势项位移与波动项位移,利用多项式拟合预测趋势项位移;然后,通过灰色关联度筛选外界诱发因子并运用LSTM模型预测波动项位移;最后,叠加周期项位移与波动项位移,得到累计位移。以新滩滑坡为例,并与(recurrent neural network,RNN)模型以及传统静态神经网络模型BP、ELM进行对比分析,采用平均百分比误差(MAPE),均方根误差(RMSE),拟合优度(R 2)分别对其进行评价。应用结果表明:相比于传统静态模型,LSTM与RNN均适用于滑坡位移动态预测;对比结果显示,LSTM模型具有较好的预测精度,MAPE与RMSE分别为1.026%、0.327 mm,拟合优度R 2为0.978。展开更多
Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicabilit...Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.展开更多
Network texts have become important carriers of cybersecurity information on the Internet.These texts include the latest security events such as vulnerability exploitations,attack discoveries,advanced persistent threa...Network texts have become important carriers of cybersecurity information on the Internet.These texts include the latest security events such as vulnerability exploitations,attack discoveries,advanced persistent threats,and so on.Extracting cybersecurity entities from these unstructured texts is a critical and fundamental task in many cybersecurity applications.However,most Named Entity Recognition(NER)models are suitable only for general fields,and there has been little research focusing on cybersecurity entity extraction in the security domain.To this end,in this paper,we propose a novel cybersecurity entity identification model based on Bidirectional Long Short-Term Memory with Conditional Random Fields(Bi-LSTM with CRF)to extract security-related concepts and entities from unstructured text.This model,which we have named XBi LSTM-CRF,consists of a word-embedding layer,a bidirectional LSTM layer,and a CRF layer,and concatenates X input with bidirectional LSTM output.Via extensive experiments on an open-source dataset containing an office security bulletin,security blogs,and the Common Vulnerabilities and Exposures list,we demonstrate that XBi LSTM-CRF achieves better cybersecurity entity extraction than state-of-the-art models.展开更多
Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning netwo...Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.展开更多
The global financial and economic market is now made up of several structures that are powerful and complex.In the last few decades,a few techniques and theories have been implemented that have revolutionized the unde...The global financial and economic market is now made up of several structures that are powerful and complex.In the last few decades,a few techniques and theories have been implemented that have revolutionized the understanding of those systems to forecast financial markets based on time series analysis.However still,none has been shown to function successfully consistently.In this project,a special form of Neural Network Modeling called LSTM to forecast the foreign exchange rate of currencies.In several different forecasting applications,this method of modelling has become popular as it can be defined complex non-linear relationships between variables and the outcome it wishes to predict.In compare to the stock market,exchange rates tend to be more relevant due to the availability of macroeconomic data that can be used to train the network to learn the impact of particular variables on the rate to be predicted.The information was collected using Quandl,an economic and financial platform that offers quantitative indicators for a wide variety of countries.Model is compared with three different metrics by exponential moving average and an autoregressive integrated moving average.then compare and validate the ability of the model to reliably predict future values and compare which of the models predicted the most correctly.展开更多
There are many techniques using sensors and wearable devices for detecting and monitoring patients with Parkinson’s disease(PD).A recent development is the utilization of human interaction with computer keyboards for...There are many techniques using sensors and wearable devices for detecting and monitoring patients with Parkinson’s disease(PD).A recent development is the utilization of human interaction with computer keyboards for analyzing and identifying motor signs in the early stages of the disease.Current designs for classification of time series of computer-key hold durations recorded from healthy control and PD subjects require the time series of length to be considerably long.With an attempt to avoid discomfort to participants in performing long physical tasks for data recording,this paper introduces the use of fuzzy recurrence plots of very short time series as input data for the machine training and classification with long short-term memory(LSTM)neural networks.Being an original approach that is able to both significantly increase the feature dimensions and provides the property of deterministic dynamical systems of very short time series for information processing carried out by an LSTM layer architecture,fuzzy recurrence plots provide promising results and outperform the direct input of the time series for the classification of healthy control and early PD subjects.展开更多
Tunnel boring machines(TBMs)are widely used in tunnel engineering because of their safety and efficiency.The TBM penetration rate(PR)is crucial,as its real-time prediction can reflect the adaptation of a TBM under cur...Tunnel boring machines(TBMs)are widely used in tunnel engineering because of their safety and efficiency.The TBM penetration rate(PR)is crucial,as its real-time prediction can reflect the adaptation of a TBM under current geological conditions and assist the adjustment of operating parameters.In this study,deep learning technology is applied to TBM performance prediction,and a PR prediction model based on a long short-term memory(LSTM)neuron network is proposed.To verify the performance of the proposed model,the machine parameters,rock mass parameters,and geological survey data from the water conveyance tunnel of the Hangzhou Second Water Source project were collected to form a dataset.Furthermore,2313 excavation cycles were randomly composed of training datasets to train the LSTM-based model,and 257 excavation cycles were used as a testing dataset to test the performance.The root mean square error and the mean absolute error of the proposed model are 4.733 and 3.204,respectively.Compared with Recurrent neuron network(RNN)based model and traditional time-series prediction model autoregressive integrated moving average with explanation variables(ARIMAX),the overall performance on proposed model is better.Moreover,in the rapidly increasing period of the PR,the error of the LSTM-based model prediction curve is significantly smaller than those of the other two models.The prediction results indicate that the LSTM-based model proposed herein is relatively accurate,thereby providing guidance for the excavation process of TBMs and offering practical application value.展开更多
Turbofan engine is a critical aircraft component with complex structure and high-reliability requirements. Effectively predicting the remaining useful life(RUL) of turbofan engines has essential significance for devel...Turbofan engine is a critical aircraft component with complex structure and high-reliability requirements. Effectively predicting the remaining useful life(RUL) of turbofan engines has essential significance for developing maintenance strategies and reducing maintenance costs. Considering the characteristics of large sample size and high dimension of monitoring data, a hybrid health condition prediction model integrating the advantages of autoencoder and bidirectional long short-term memory(BLSTM) is proposed to improve the prediction accuracy of RUL. Autoencoder is used as a feature extractor to compress condition monitoring data. BLSTM is designed to capture the bidirectional long-range dependencies of features. A hybrid deep learning prediction model of RUL is constructed. This model has been tested on a benchmark dataset. The results demonstrate that this autoencoder-BLSTM hybrid model has a better prediction accuracy than the existing methods, such as multi-layer perceptron(MLP), support vector regression(SVR), convolutional neural network(CNN) and long short-term memory(LSTM). The proposed model can provide strong support for the health management and maintenance strategy development of turbofan engines.展开更多
Short-term traffic flow forecasting is a significant part of intelligent transportation system.In some traffic control scenarios,obtaining future traffic flow in advance is conducive to highway management department t...Short-term traffic flow forecasting is a significant part of intelligent transportation system.In some traffic control scenarios,obtaining future traffic flow in advance is conducive to highway management department to have sufficient time to formulate corresponding traffic flow control measures.In hence,it is meaningful to establish an accurate short-term traffic flow method and provide reference for peak traffic flow warning.This paper proposed a new hybrid model for traffic flow forecasting,which is composed of the variational mode decomposition(VMD)method,the group method of data handling(GMDH)neural network,bi-directional long and short term memory(BILSTM)network and ELMAN network,and is optimized by the imperialist competitive algorithm(ICA)method.To illustrate the performance of the proposed model,there are several comparative experiments between the proposed model and other models.The experiment results show that 1)BILSTM network,GMDH network and ELMAN network have better predictive performance than other single models;2)VMD can significantly improve the predictive performance of the ICA-GMDH-BILSTM-ELMAN model.The effect of VMD method is better than that of EEMD method and FEEMD method.To conclude,the proposed model which is made up of the VMD method,the ICA method,the BILSTM network,the GMDH network and the ELMAN network has excellent predictive ability for traffic flow series.展开更多
With the application of artificial intelligence technology in the power industry,the knowledge graph is expected to play a key role in power grid dispatch processes,intelligent maintenance,and customer service respons...With the application of artificial intelligence technology in the power industry,the knowledge graph is expected to play a key role in power grid dispatch processes,intelligent maintenance,and customer service response provision.Knowledge graphs are usually constructed based on entity recognition.Specifically,based on the mining of entity attributes and relationships,domain knowledge graphs can be constructed through knowledge fusion.In this work,the entities and characteristics of power entity recognition are analyzed,the mechanism of entity recognition is clarified,and entity recognition techniques are analyzed in the context of the power domain.Power entity recognition based on the conditional random fields (CRF) and bidirectional long short-term memory (BLSTM) models is investigated,and the two methods are comparatively analyzed.The results indicated that the CRF model,with an accuracy of 83%,can better identify the power entities compared to the BLSTM.The CRF approach can thus be applied to the entity extraction for knowledge graph construction in the power field.展开更多
Short-term water demand forecasting provides guidance on real-time water allocation in the water supply network, which help water utilities reduce energy cost and avoid potential accidents. Although a variety of metho...Short-term water demand forecasting provides guidance on real-time water allocation in the water supply network, which help water utilities reduce energy cost and avoid potential accidents. Although a variety of methods have been proposed to improve forecast accuracy, it is still difficult for statistical models to learn the periodic patterns due to the chaotic nature of the water demand data with high temporal resolution. To overcome this issue from the perspective of improving data predictability, we proposed a hybrid Wavelet-CNN-LSTM model, that combines time-frequency decomposition characteristics of Wavelet Multi-Resolution Analysis (MRA) and implement it into an advanced deep learning model, CNN-LSTM. Four models - ANN, Conv1D, LSTM, GRUN - are used to compare with Wavelet-CNN-LSTM, and the results show that Wavelet-CNN-LSTM outperforms the other models both in single-step and multi-steps prediction. Besides, further mechanistic analysis revealed that MRA produce significant effect on improving model accuracy.展开更多
As typical prosumers,commercial buildings equipped with electric vehicle(EV)charging piles and solar photovoltaic panels require an effective energy management method.However,the conventional optimization-model-based ...As typical prosumers,commercial buildings equipped with electric vehicle(EV)charging piles and solar photovoltaic panels require an effective energy management method.However,the conventional optimization-model-based building energy management system faces significant challenges regarding prediction and calculation in online execution.To address this issue,a long short-term memory(LSTM)recurrent neural network(RNN)based machine learning algorithm is proposed in this paper to schedule the charging and discharging of numerous EVs in commercial-building prosumers.Under the proposed system control structure,the LSTM algorithm can be separated into offline and online stages.At the offline stage,the LSTM is used to map states(inputs)to decisions(outputs)based on the network training.At the online stage,once the current state is input,the LSTM can quickly generate a solution without any additional prediction.A preliminary data processing rule and an additional output filtering procedure are designed to improve the decision performance of LSTM network.The simulation results demonstrate that the LSTM algorithm can generate near-optimal solutions in milliseconds and significantly reduce the prediction and calculation pressures compared with the conventional optimization algorithm.展开更多
Anomaly detection is crucial to the flight safety and maintenance of unmanned aerial vehicles(UAVs)and has attracted extensive attention from scholars.Knowledge-based approaches rely on prior knowledge,while model-bas...Anomaly detection is crucial to the flight safety and maintenance of unmanned aerial vehicles(UAVs)and has attracted extensive attention from scholars.Knowledge-based approaches rely on prior knowledge,while model-based approaches are challenging for constructing accurate and complex physical models of unmanned aerial systems(UASs).Although data-driven methods do not require extensive prior knowledge and accurate physical UAS models,they often lack parameter selection and are limited by the cost of labeling anomalous data.Furthermore,flight data with random noise pose a significant challenge for anomaly detection.This work proposes a spatiotemporal correlation based on long short-term memory and autoencoder(STCLSTM-AE)neural network data-driven method for unsupervised anomaly detection and recovery of UAV flight data.First,UAV flight data are preprocessed by combining the Savitzky-Golay filter data processing technique to mitigate the effect of noise in the original historical flight data on the model.Correlation-based feature subset selection is subsequently performed to reduce the reliance on expert knowledge.Then,the extracted features are used as the input of the designed LSTM-AE model to achieve the anomaly detection and recovery of UAV flight data in an unsupervised manner.Finally,the method's effectiveness is validated on real UAV flight data.展开更多
To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with...To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.展开更多
Accurately forecasting the nonlinear degradation of lithium-ion batteries(LIBs)using early-cycle data can obviously shorten the battery test time,which accelerates battery optimization and production.In this work,a se...Accurately forecasting the nonlinear degradation of lithium-ion batteries(LIBs)using early-cycle data can obviously shorten the battery test time,which accelerates battery optimization and production.In this work,a self-adaptive long short-term memory(SA-LSTM)method has been proposed to predict the battery degradation trajectory and battery lifespan with only early cycling data.Specifically,two features were extracted from discharge voltage curves by a time-series-based approach and forecasted to further cycles using SA-LSTM model.The as-obtained features were correlated with the capacity to predict the capacity degradation trajectory by generalized multiple linear regression model.The proposed method achieved an average online prediction error of 6.00%and 6.74%for discharge capacity and end of life,respectively,when using the early-cycle discharge information until 90%capacity retention.Fur-thermore,the importance of temperature control was highlighted by correlat-ing the features with the average temperature in each cycle.This work develops a self-adaptive data-driven method to accurately predict the cycling life of LIBs,and unveils the underlying degradation mechanism and the impor-tance of controlling environmental temperature.展开更多
The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning mode...The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning models have some problems,such as poor nonlinear performance,local optimum and incomplete factors feature extraction.These issues can affect the accuracy of slope stability prediction.Therefore,a deep learning algorithm called Long short-term memory(LSTM)has been innovatively proposed to predict slope stability.Taking the Ganzhou City in China as the study area,the landslide inventory and their characteristics of geotechnical parameters,slope height and slope angle are analyzed.Based on these characteristics,typical soil slopes are constructed using the Geo-Studio software.Five control factors affecting slope stability,including slope height,slope angle,internal friction angle,cohesion and volumetric weight,are selected to form different slope and construct model input variables.Then,the limit equilibrium method is used to calculate the stability coefficients of these typical soil slopes under different control factors.Each slope stability coefficient and its corresponding control factors is a slope sample.As a result,a total of 2160 training samples and 450 testing samples are constructed.These sample sets are imported into LSTM for modelling and compared with the support vector machine(SVM),random forest(RF)and convo-lutional neural network(CNN).The results show that the LSTM overcomes the problem that the commonly used machine learning models have difficulty extracting global features.Furthermore,LSTM has a better prediction performance for slope stability compared to SVM,RF and CNN models.展开更多
Speed forecasting has numerous applications in intelligent transport systems’design and control,especially for safety and road efficiency applications.In the field of electromobility,it represents the most dynamic pa...Speed forecasting has numerous applications in intelligent transport systems’design and control,especially for safety and road efficiency applications.In the field of electromobility,it represents the most dynamic parameter for efficient online in-vehicle energy management.However,vehicles’speed forecasting is a challenging task,because its estimation is closely related to various features,which can be classified into two categories,endogenous and exogenous features.Endogenous features represent electric vehicles’characteristics,whereas exogenous ones represent its surrounding context,such as traffic,weather,and road conditions.In this paper,a speed forecasting method based on the Long Short-Term Memory(LSTM)is introduced.The LSTM model training is performed upon a dataset collected from a traffic simulator based on real-world data representing urban itineraries.The proposed models are generated for univariate and multivariate scenarios and are assessed in terms of accuracy for speed forecasting.Simulation results show that the multivariate model outperforms the univariate model for short-and long-term forecasting.展开更多
In recent years,e-sports has rapidly developed,and the industry has produced large amounts of data with specifications,and these data are easily to be obtained.Due to the above characteristics,data mining and deep lea...In recent years,e-sports has rapidly developed,and the industry has produced large amounts of data with specifications,and these data are easily to be obtained.Due to the above characteristics,data mining and deep learning methods can be used to guide players and develop appropriate strategies to win games.As one of the world’s most famous e-sports events,Dota2 has a large audience base and a good game system.A victory in a game is often associated with a hero’s match,and players are often unable to pick the best lineup to compete.To solve this problem,in this paper,we present an improved bidirectional Long Short-Term Memory(LSTM)neural network model for Dota2 lineup recommendations.The model uses the Continuous Bag Of Words(CBOW)model in the Word2 vec model to generate hero vectors.The CBOW model can predict the context of a word in a sentence.Accordingly,a word is transformed into a hero,a sentence into a lineup,and a word vector into a hero vector,the model applied in this article recommends the last hero according to the first four heroes selected first,thereby solving a series of recommendation problems.展开更多
Electric load forecasting is an efficient tool for system planning, and consequently, building sustainable power systems. However, achieving desirable performance is difficult owing to the irregular, nonstationary, no...Electric load forecasting is an efficient tool for system planning, and consequently, building sustainable power systems. However, achieving desirable performance is difficult owing to the irregular, nonstationary, nonlinear, and noisy nature of the observed data. Therefore, a new attention-based encoderdecoder model is proposed, called empirical mode decomposition-attention-long short-term memory(EMD-Att-LSTM).EMD is a data-driven technique used for the decomposition of complex series into subsequent simpler series. It explores the inherent properties of data to obtain the components such as trend and seasonality. Neural network architecture driven by deep learning uses the idea of a fine-grained attention mechanism, that is, considering the hidden state instead of the hidden state vectors, which can help reflect the significance and contributions of each hidden state dimension. In addition, it is useful for locating and concentrating the relevant temporary data,leading to a distinctly interpretable network. To evaluate the proposed model, we use the repository dataset of Australian energy market operator(AEMO). The proposed architecture provides superior empirical results compared with other advanced models. It is explored using the indices of root mean square error(RMSE) and mean absolute percentage error(MAPE).展开更多
文摘针对传统滑坡位移预测模型存在对历史数据遗忘的问题,提出了一种基于长短时记忆(long short time memory,LSTM)网络的滑坡位移动态预测模型。首先,将滑坡累计位移分解为趋势项位移与波动项位移,利用多项式拟合预测趋势项位移;然后,通过灰色关联度筛选外界诱发因子并运用LSTM模型预测波动项位移;最后,叠加周期项位移与波动项位移,得到累计位移。以新滩滑坡为例,并与(recurrent neural network,RNN)模型以及传统静态神经网络模型BP、ELM进行对比分析,采用平均百分比误差(MAPE),均方根误差(RMSE),拟合优度(R 2)分别对其进行评价。应用结果表明:相比于传统静态模型,LSTM与RNN均适用于滑坡位移动态预测;对比结果显示,LSTM模型具有较好的预测精度,MAPE与RMSE分别为1.026%、0.327 mm,拟合优度R 2为0.978。
基金supported by the Natural Science Foundation of China(Grant Nos.51979158,51639008,51679135,and 51422905)the Program of Shanghai Academic Research Leader by Science and Technology Commission of Shanghai Municipality(Project No.19XD1421900)。
文摘Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.
基金supported by the National Natural Science Foundation of China(Nos.61702508,61802404,and U1836209)the National Key Research and Development Program of China(Nos.2018YFB0803602 and 2016QY06X1204)+2 种基金the National Social Science Foundation of China(No.19BSH022)supported by the Key Laboratory of Network Assessment Technology,Chinese Academy of SciencesBeijing Key Laboratory of Network Security and Protection Technology。
文摘Network texts have become important carriers of cybersecurity information on the Internet.These texts include the latest security events such as vulnerability exploitations,attack discoveries,advanced persistent threats,and so on.Extracting cybersecurity entities from these unstructured texts is a critical and fundamental task in many cybersecurity applications.However,most Named Entity Recognition(NER)models are suitable only for general fields,and there has been little research focusing on cybersecurity entity extraction in the security domain.To this end,in this paper,we propose a novel cybersecurity entity identification model based on Bidirectional Long Short-Term Memory with Conditional Random Fields(Bi-LSTM with CRF)to extract security-related concepts and entities from unstructured text.This model,which we have named XBi LSTM-CRF,consists of a word-embedding layer,a bidirectional LSTM layer,and a CRF layer,and concatenates X input with bidirectional LSTM output.Via extensive experiments on an open-source dataset containing an office security bulletin,security blogs,and the Common Vulnerabilities and Exposures list,we demonstrate that XBi LSTM-CRF achieves better cybersecurity entity extraction than state-of-the-art models.
文摘Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.
文摘The global financial and economic market is now made up of several structures that are powerful and complex.In the last few decades,a few techniques and theories have been implemented that have revolutionized the understanding of those systems to forecast financial markets based on time series analysis.However still,none has been shown to function successfully consistently.In this project,a special form of Neural Network Modeling called LSTM to forecast the foreign exchange rate of currencies.In several different forecasting applications,this method of modelling has become popular as it can be defined complex non-linear relationships between variables and the outcome it wishes to predict.In compare to the stock market,exchange rates tend to be more relevant due to the availability of macroeconomic data that can be used to train the network to learn the impact of particular variables on the rate to be predicted.The information was collected using Quandl,an economic and financial platform that offers quantitative indicators for a wide variety of countries.Model is compared with three different metrics by exponential moving average and an autoregressive integrated moving average.then compare and validate the ability of the model to reliably predict future values and compare which of the models predicted the most correctly.
文摘There are many techniques using sensors and wearable devices for detecting and monitoring patients with Parkinson’s disease(PD).A recent development is the utilization of human interaction with computer keyboards for analyzing and identifying motor signs in the early stages of the disease.Current designs for classification of time series of computer-key hold durations recorded from healthy control and PD subjects require the time series of length to be considerably long.With an attempt to avoid discomfort to participants in performing long physical tasks for data recording,this paper introduces the use of fuzzy recurrence plots of very short time series as input data for the machine training and classification with long short-term memory(LSTM)neural networks.Being an original approach that is able to both significantly increase the feature dimensions and provides the property of deterministic dynamical systems of very short time series for information processing carried out by an LSTM layer architecture,fuzzy recurrence plots provide promising results and outperform the direct input of the time series for the classification of healthy control and early PD subjects.
基金supported by National Natural Science Foundation of China(No.51739007)the National Science Fund for Excellent Young Scholars(No.51922067)+3 种基金Joint Funds of the National Natural Science Foundation of China(No.U1806226)Taishan Scholars Program of Shandong Province(tsqn20190900,tsqn201909044)the Key Research and Development Program of Shandong Province(No.Z135050009107)the Interdisciplinary Development Program of Shandong University(No.2017JC002).
文摘Tunnel boring machines(TBMs)are widely used in tunnel engineering because of their safety and efficiency.The TBM penetration rate(PR)is crucial,as its real-time prediction can reflect the adaptation of a TBM under current geological conditions and assist the adjustment of operating parameters.In this study,deep learning technology is applied to TBM performance prediction,and a PR prediction model based on a long short-term memory(LSTM)neuron network is proposed.To verify the performance of the proposed model,the machine parameters,rock mass parameters,and geological survey data from the water conveyance tunnel of the Hangzhou Second Water Source project were collected to form a dataset.Furthermore,2313 excavation cycles were randomly composed of training datasets to train the LSTM-based model,and 257 excavation cycles were used as a testing dataset to test the performance.The root mean square error and the mean absolute error of the proposed model are 4.733 and 3.204,respectively.Compared with Recurrent neuron network(RNN)based model and traditional time-series prediction model autoregressive integrated moving average with explanation variables(ARIMAX),the overall performance on proposed model is better.Moreover,in the rapidly increasing period of the PR,the error of the LSTM-based model prediction curve is significantly smaller than those of the other two models.The prediction results indicate that the LSTM-based model proposed herein is relatively accurate,thereby providing guidance for the excavation process of TBMs and offering practical application value.
基金the National Natural Science Foundation of China(Nos.51505288 and 51875359)the TBT Project of Shanghai(No.18TBT003)the Project of Shanghai Telecom(No.17C1ZA0069SH301)
文摘Turbofan engine is a critical aircraft component with complex structure and high-reliability requirements. Effectively predicting the remaining useful life(RUL) of turbofan engines has essential significance for developing maintenance strategies and reducing maintenance costs. Considering the characteristics of large sample size and high dimension of monitoring data, a hybrid health condition prediction model integrating the advantages of autoencoder and bidirectional long short-term memory(BLSTM) is proposed to improve the prediction accuracy of RUL. Autoencoder is used as a feature extractor to compress condition monitoring data. BLSTM is designed to capture the bidirectional long-range dependencies of features. A hybrid deep learning prediction model of RUL is constructed. This model has been tested on a benchmark dataset. The results demonstrate that this autoencoder-BLSTM hybrid model has a better prediction accuracy than the existing methods, such as multi-layer perceptron(MLP), support vector regression(SVR), convolutional neural network(CNN) and long short-term memory(LSTM). The proposed model can provide strong support for the health management and maintenance strategy development of turbofan engines.
基金Project(61873283)supported by the National Natural Science Foundation of ChinaProject(KQ1707017)supported by the Changsha Science&Technology Project,ChinaProject(2019CX005)supported by the Innovation Driven Project of the Central South University,China。
文摘Short-term traffic flow forecasting is a significant part of intelligent transportation system.In some traffic control scenarios,obtaining future traffic flow in advance is conducive to highway management department to have sufficient time to formulate corresponding traffic flow control measures.In hence,it is meaningful to establish an accurate short-term traffic flow method and provide reference for peak traffic flow warning.This paper proposed a new hybrid model for traffic flow forecasting,which is composed of the variational mode decomposition(VMD)method,the group method of data handling(GMDH)neural network,bi-directional long and short term memory(BILSTM)network and ELMAN network,and is optimized by the imperialist competitive algorithm(ICA)method.To illustrate the performance of the proposed model,there are several comparative experiments between the proposed model and other models.The experiment results show that 1)BILSTM network,GMDH network and ELMAN network have better predictive performance than other single models;2)VMD can significantly improve the predictive performance of the ICA-GMDH-BILSTM-ELMAN model.The effect of VMD method is better than that of EEMD method and FEEMD method.To conclude,the proposed model which is made up of the VMD method,the ICA method,the BILSTM network,the GMDH network and the ELMAN network has excellent predictive ability for traffic flow series.
基金supported by Science and Technology Project of State Grid Corporation(Research and Application of Intelligent Energy Meter Quality Analysis and Evaluation Technology Based on Full Chain Data)
文摘With the application of artificial intelligence technology in the power industry,the knowledge graph is expected to play a key role in power grid dispatch processes,intelligent maintenance,and customer service response provision.Knowledge graphs are usually constructed based on entity recognition.Specifically,based on the mining of entity attributes and relationships,domain knowledge graphs can be constructed through knowledge fusion.In this work,the entities and characteristics of power entity recognition are analyzed,the mechanism of entity recognition is clarified,and entity recognition techniques are analyzed in the context of the power domain.Power entity recognition based on the conditional random fields (CRF) and bidirectional long short-term memory (BLSTM) models is investigated,and the two methods are comparatively analyzed.The results indicated that the CRF model,with an accuracy of 83%,can better identify the power entities compared to the BLSTM.The CRF approach can thus be applied to the entity extraction for knowledge graph construction in the power field.
基金financially supported by the National Natural Science Foundation of China(No.51978494)the Science and Technology Innovation Program Project of Shanghai City Investment Co.,Ltd.(No.CTKY-ZDXM-2020-012).
文摘Short-term water demand forecasting provides guidance on real-time water allocation in the water supply network, which help water utilities reduce energy cost and avoid potential accidents. Although a variety of methods have been proposed to improve forecast accuracy, it is still difficult for statistical models to learn the periodic patterns due to the chaotic nature of the water demand data with high temporal resolution. To overcome this issue from the perspective of improving data predictability, we proposed a hybrid Wavelet-CNN-LSTM model, that combines time-frequency decomposition characteristics of Wavelet Multi-Resolution Analysis (MRA) and implement it into an advanced deep learning model, CNN-LSTM. Four models - ANN, Conv1D, LSTM, GRUN - are used to compare with Wavelet-CNN-LSTM, and the results show that Wavelet-CNN-LSTM outperforms the other models both in single-step and multi-steps prediction. Besides, further mechanistic analysis revealed that MRA produce significant effect on improving model accuracy.
基金This work was supported by the National Natural Science Foundation of China(No.51877078)the State Key Laboratory of Smart Grid Protection and Operation Control Open Project(No.SGNR0000KJJS1907535)the Beijing Nova Program(No.Z201100006820106)。
文摘As typical prosumers,commercial buildings equipped with electric vehicle(EV)charging piles and solar photovoltaic panels require an effective energy management method.However,the conventional optimization-model-based building energy management system faces significant challenges regarding prediction and calculation in online execution.To address this issue,a long short-term memory(LSTM)recurrent neural network(RNN)based machine learning algorithm is proposed in this paper to schedule the charging and discharging of numerous EVs in commercial-building prosumers.Under the proposed system control structure,the LSTM algorithm can be separated into offline and online stages.At the offline stage,the LSTM is used to map states(inputs)to decisions(outputs)based on the network training.At the online stage,once the current state is input,the LSTM can quickly generate a solution without any additional prediction.A preliminary data processing rule and an additional output filtering procedure are designed to improve the decision performance of LSTM network.The simulation results demonstrate that the LSTM algorithm can generate near-optimal solutions in milliseconds and significantly reduce the prediction and calculation pressures compared with the conventional optimization algorithm.
基金supported by the National Key Research and Development Program of China(Grant No.2020YFB1713300)the Guizhou Provincial Colleges and Universities Talent Training Base Project(Grant No.[2020]009)+3 种基金the Guizhou Province Science and Technology Plan Project(Grant Nos.[2015]4011,[2017]5788)the Guizhou Provincial Department of Education Youth Science and Technology Talent Growth Project(Grant No.[2022]142)the Scientific Research Project for Introducing Talents from Guizhou University(Grant No.(2021)74)the Guizhou Province Higher Education Integrated Research Platform Project(Grant No.[2020]005)。
文摘Anomaly detection is crucial to the flight safety and maintenance of unmanned aerial vehicles(UAVs)and has attracted extensive attention from scholars.Knowledge-based approaches rely on prior knowledge,while model-based approaches are challenging for constructing accurate and complex physical models of unmanned aerial systems(UASs).Although data-driven methods do not require extensive prior knowledge and accurate physical UAS models,they often lack parameter selection and are limited by the cost of labeling anomalous data.Furthermore,flight data with random noise pose a significant challenge for anomaly detection.This work proposes a spatiotemporal correlation based on long short-term memory and autoencoder(STCLSTM-AE)neural network data-driven method for unsupervised anomaly detection and recovery of UAV flight data.First,UAV flight data are preprocessed by combining the Savitzky-Golay filter data processing technique to mitigate the effect of noise in the original historical flight data on the model.Correlation-based feature subset selection is subsequently performed to reduce the reliance on expert knowledge.Then,the extracted features are used as the input of the designed LSTM-AE model to achieve the anomaly detection and recovery of UAV flight data in an unsupervised manner.Finally,the method's effectiveness is validated on real UAV flight data.
基金The National Key R&D Program of China under contract No.2016YFC1402103
文摘To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.
基金supported by the National Key Research and Development Program(2021YFB2500300)Beijing Municipal Natural Science Foundation(Z200011)+1 种基金National Natural Science Foundation of China(T2322015,22209093,22209094,22379121,and 21825501)the Fundamental Research Funds for the Central Universities.
文摘Accurately forecasting the nonlinear degradation of lithium-ion batteries(LIBs)using early-cycle data can obviously shorten the battery test time,which accelerates battery optimization and production.In this work,a self-adaptive long short-term memory(SA-LSTM)method has been proposed to predict the battery degradation trajectory and battery lifespan with only early cycling data.Specifically,two features were extracted from discharge voltage curves by a time-series-based approach and forecasted to further cycles using SA-LSTM model.The as-obtained features were correlated with the capacity to predict the capacity degradation trajectory by generalized multiple linear regression model.The proposed method achieved an average online prediction error of 6.00%and 6.74%for discharge capacity and end of life,respectively,when using the early-cycle discharge information until 90%capacity retention.Fur-thermore,the importance of temperature control was highlighted by correlat-ing the features with the average temperature in each cycle.This work develops a self-adaptive data-driven method to accurately predict the cycling life of LIBs,and unveils the underlying degradation mechanism and the impor-tance of controlling environmental temperature.
基金funded by the National Natural Science Foundation of China (41807285)。
文摘The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning models have some problems,such as poor nonlinear performance,local optimum and incomplete factors feature extraction.These issues can affect the accuracy of slope stability prediction.Therefore,a deep learning algorithm called Long short-term memory(LSTM)has been innovatively proposed to predict slope stability.Taking the Ganzhou City in China as the study area,the landslide inventory and their characteristics of geotechnical parameters,slope height and slope angle are analyzed.Based on these characteristics,typical soil slopes are constructed using the Geo-Studio software.Five control factors affecting slope stability,including slope height,slope angle,internal friction angle,cohesion and volumetric weight,are selected to form different slope and construct model input variables.Then,the limit equilibrium method is used to calculate the stability coefficients of these typical soil slopes under different control factors.Each slope stability coefficient and its corresponding control factors is a slope sample.As a result,a total of 2160 training samples and 450 testing samples are constructed.These sample sets are imported into LSTM for modelling and compared with the support vector machine(SVM),random forest(RF)and convo-lutional neural network(CNN).The results show that the LSTM overcomes the problem that the commonly used machine learning models have difficulty extracting global features.Furthermore,LSTM has a better prediction performance for slope stability compared to SVM,RF and CNN models.
基金supported by MIGRID project(No.5-398,2017–2019),which was funded by USAID under the PEER program
文摘Speed forecasting has numerous applications in intelligent transport systems’design and control,especially for safety and road efficiency applications.In the field of electromobility,it represents the most dynamic parameter for efficient online in-vehicle energy management.However,vehicles’speed forecasting is a challenging task,because its estimation is closely related to various features,which can be classified into two categories,endogenous and exogenous features.Endogenous features represent electric vehicles’characteristics,whereas exogenous ones represent its surrounding context,such as traffic,weather,and road conditions.In this paper,a speed forecasting method based on the Long Short-Term Memory(LSTM)is introduced.The LSTM model training is performed upon a dataset collected from a traffic simulator based on real-world data representing urban itineraries.The proposed models are generated for univariate and multivariate scenarios and are assessed in terms of accuracy for speed forecasting.Simulation results show that the multivariate model outperforms the univariate model for short-and long-term forecasting.
基金the Guangdong Province Key Research and Development Plan(No.2019B010137004)the National Natural Science Foundation of China(Nos.61402149 and 61871140)+3 种基金the Scientific and Technological Project of Henan Province(Nos.182102110065,182102210238,and 202102310340)the Natural Science Foundation of Henan Educational Committee(No.17B520006)Guangdong Province Universities and Colleges Pearl River Scholar Funded Scheme(2019)Foundation of University Young Key Teacher of Henan Province(No.2019GGJS040)。
文摘In recent years,e-sports has rapidly developed,and the industry has produced large amounts of data with specifications,and these data are easily to be obtained.Due to the above characteristics,data mining and deep learning methods can be used to guide players and develop appropriate strategies to win games.As one of the world’s most famous e-sports events,Dota2 has a large audience base and a good game system.A victory in a game is often associated with a hero’s match,and players are often unable to pick the best lineup to compete.To solve this problem,in this paper,we present an improved bidirectional Long Short-Term Memory(LSTM)neural network model for Dota2 lineup recommendations.The model uses the Continuous Bag Of Words(CBOW)model in the Word2 vec model to generate hero vectors.The CBOW model can predict the context of a word in a sentence.Accordingly,a word is transformed into a hero,a sentence into a lineup,and a word vector into a hero vector,the model applied in this article recommends the last hero according to the first four heroes selected first,thereby solving a series of recommendation problems.
文摘Electric load forecasting is an efficient tool for system planning, and consequently, building sustainable power systems. However, achieving desirable performance is difficult owing to the irregular, nonstationary, nonlinear, and noisy nature of the observed data. Therefore, a new attention-based encoderdecoder model is proposed, called empirical mode decomposition-attention-long short-term memory(EMD-Att-LSTM).EMD is a data-driven technique used for the decomposition of complex series into subsequent simpler series. It explores the inherent properties of data to obtain the components such as trend and seasonality. Neural network architecture driven by deep learning uses the idea of a fine-grained attention mechanism, that is, considering the hidden state instead of the hidden state vectors, which can help reflect the significance and contributions of each hidden state dimension. In addition, it is useful for locating and concentrating the relevant temporary data,leading to a distinctly interpretable network. To evaluate the proposed model, we use the repository dataset of Australian energy market operator(AEMO). The proposed architecture provides superior empirical results compared with other advanced models. It is explored using the indices of root mean square error(RMSE) and mean absolute percentage error(MAPE).
