Hyperparameters are important for machine learning algorithms since they directly control the behaviors of training algorithms and have a significant effect on the performance of machine learning models. Several techn...Hyperparameters are important for machine learning algorithms since they directly control the behaviors of training algorithms and have a significant effect on the performance of machine learning models. Several techniques have been developed and successfully applied for certain application domains. However, this work demands professional knowledge and expert experience. And sometimes it has to resort to the brute-force search.Therefore, if an efficient hyperparameter optimization algorithm can be developed to optimize any given machine learning method, it will greatly improve the efficiency of machine learning. In this paper, we consider building the relationship between the performance of the machine learning models and their hyperparameters by Gaussian processes. In this way, the hyperparameter tuning problem can be abstracted as an optimization problem and Bayesian optimization is used to solve the problem. Bayesian optimization is based on the Bayesian theorem. It sets a prior over the optimization function and gathers the information from the previous sample to update the posterior of the optimization function. A utility function selects the next sample point to maximize the optimization function.Several experiments were conducted on standard test datasets. Experiment results show that the proposed method can find the best hyperparameters for the widely used machine learning models, such as the random forest algorithm and the neural networks, even multi-grained cascade forest under the consideration of time cost.展开更多
堆石坝变形监测数据是一种时间序列数据,可以用时序预测模型挖掘其规律并进行预测。本文利用时序预测模型提出一种堆石坝变形预测方法,该方法首先采用时间序列分解(seasonal-trend decomposition procedure based on loess,STL)将堆石...堆石坝变形监测数据是一种时间序列数据,可以用时序预测模型挖掘其规律并进行预测。本文利用时序预测模型提出一种堆石坝变形预测方法,该方法首先采用时间序列分解(seasonal-trend decomposition procedure based on loess,STL)将堆石坝变形监测数据分解为趋势项、周期项和不规则波动三部分,再使用经验模态分解(empirical mode decomposition,EMD)对不规则波动平稳化处理,最后利用长短期记忆网络(long short-term memory,LSTM)预测分解后的序列,并利用贝叶斯优化方法进行超参数优化。为评估该方法的预测效果,以水布垭面板堆石坝为例,通过控制训练时长、预测时长、离群值数目等变量进行多组仿真实验,并与其他时序预测模型对比。结果表明该方法预测精度较高,适用性较广,对于堆石坝的性状评估具有一定的应用价值。展开更多
基金supported in part by the National Natural Science Foundation of China under Grant No.61503059
文摘Hyperparameters are important for machine learning algorithms since they directly control the behaviors of training algorithms and have a significant effect on the performance of machine learning models. Several techniques have been developed and successfully applied for certain application domains. However, this work demands professional knowledge and expert experience. And sometimes it has to resort to the brute-force search.Therefore, if an efficient hyperparameter optimization algorithm can be developed to optimize any given machine learning method, it will greatly improve the efficiency of machine learning. In this paper, we consider building the relationship between the performance of the machine learning models and their hyperparameters by Gaussian processes. In this way, the hyperparameter tuning problem can be abstracted as an optimization problem and Bayesian optimization is used to solve the problem. Bayesian optimization is based on the Bayesian theorem. It sets a prior over the optimization function and gathers the information from the previous sample to update the posterior of the optimization function. A utility function selects the next sample point to maximize the optimization function.Several experiments were conducted on standard test datasets. Experiment results show that the proposed method can find the best hyperparameters for the widely used machine learning models, such as the random forest algorithm and the neural networks, even multi-grained cascade forest under the consideration of time cost.
文摘堆石坝变形监测数据是一种时间序列数据,可以用时序预测模型挖掘其规律并进行预测。本文利用时序预测模型提出一种堆石坝变形预测方法,该方法首先采用时间序列分解(seasonal-trend decomposition procedure based on loess,STL)将堆石坝变形监测数据分解为趋势项、周期项和不规则波动三部分,再使用经验模态分解(empirical mode decomposition,EMD)对不规则波动平稳化处理,最后利用长短期记忆网络(long short-term memory,LSTM)预测分解后的序列,并利用贝叶斯优化方法进行超参数优化。为评估该方法的预测效果,以水布垭面板堆石坝为例,通过控制训练时长、预测时长、离群值数目等变量进行多组仿真实验,并与其他时序预测模型对比。结果表明该方法预测精度较高,适用性较广,对于堆石坝的性状评估具有一定的应用价值。
