QIAO Gangzhu， SU Rong， ZHANG Hongfei
（School of Data Science and Technology, North University of China, Taiyuan 030051， China）
Abstract： Time series is a kind of data widely used in various fields such as electricity forecasting, exchange rate forecasting, and solar power generation forecasting, and therefore time series prediction is of great significance. Recently, the encoder-decoder model combined with long short-term memory (LSTM) is widely used for multivariate time series prediction. However, the encoder can only encode information into fixed-length vectors, hence the performance of the model decreases rapidly as the length of the input sequence or output sequence increases. To solve this problem, we propose a combination model named AR_CLSTM based on the encoder_decoder structure and linear autoregression. The model uses a time step-based attention mechanism to enable the decoder to adaptively select past hidden states and extract useful information, and then uses convolution structure to learn the internal relationship between different dimensions of multivariate time series.In addition, AR_CLSTM combines the traditional linear autoregressive method to learn the linear relationship of the time series, so as to further reduce the error of time series prediction in the encoder_decoder structure and improve the multivariate time series Predictive effect. Experiments show that the AR_CLSTM model performs well in different time series predictions, and its root mean square error, mean square error, and average absolute error all decrease significantly.
Key words： encoder_decoder; attention mechanism; convolution; autoregression model; multivariate time series
References
［1］He Y L, Xu Q K. Overview of time series forecasting technology. Information and Communication, 2018(11)： 35-36.
［2］Mishra S, Bordin C, Taharaguchi K, et al. Comparison of deep learning models for multivariate prediction of time series wind power generation and temperature. Energy Reports, 2020, 6: 273-286.
［3］Chen Y X, Lin W W, Wang J Z. A dual-attention-based stock price trend prediction model with dual features. IEEE Access, 2019, 7: 148047-148058.
［4］Sagheer A, Kotb M. Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing, 2018, 323: 203-213.
［5］Heidari A, Khovalyg D. Short-term energy use prediction of solar-assisted water heating system: Application case of combined attention-based LSTM and time-series decomposition. Solar Energy, 2020, 207: 626-639.
［6］Liu F G, Lu Y S, Cai M Q. A hybrid method with adaptive sub-series clustering and attention-based stacked residual LSTMs for multivariate time series forecasting. IEEE Access, 2020, 8: 62423-62438.
［7］Xu X,Jeong S, Li J Q. Interpretation of electrocardiogram (ECG) rhythm by combined CNN and biLSTM. IEEE Access, 2020, 8: 125380-125388.
［8］Zhang X D, Du J H, Huang Y F, et al. Time series prediction analysis based on multi-scale hierarchical LSTM network. Computer Science, 2019, 46(s2): 52-57.
［9］Du S D, Li T R, Yang Y, et al. Multivariate time series forecasting via attention-based encoder-decoder framework. Neurocomputing, 2020, 388: 269-279
［10］Heidari A, Khovalyg D. Short-term energy use prediction of solar-assisted water heating system: Application case of combined attention-based LSTM and time-series decomposition . Solar Energy, 2020, 207: 626-639.
［11］Qin Y, Song D J, Cheng H F, et al. A dual-stage attention-based recurrent neural network for time series prediction. In: Proceedings of Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017: 2627-2633.
［12］Shih S Y, Sun F K, Lee H Y. Temporal pattern attention for multivariate time series forecasting. Machine Learning, 2019, 108 (8/9): 1421- 1441.
［13］Lai G， Chang W C， Yang Y, et al. Modeling long- and short-term temporal patterns with deep neural networks. In: Proceedings of the 41st International ACM SIGIR Conference. ACM, 2018: 95-104.
［14］Wang Z M, Zhang L, Ding Z M. Hybrid time-aligned and context attention for time series prediction. Knowledge-Based Systems, 2020, 198: 105937.
［15］Liu Y X, Duan J Y, Meng J. Difference attention based error correction LSTM model for time series prediction. Journal of Physics: Conference Series, 2020, 1550(3): 032121.
［16］Sagheer A, Kotb M . Unsupervised pre-training of a deep LSTM-based stacked autoencoder for multivariate time series forecasting problems. Scientific Reports, 2019, 9: 19038
［17］Lyu P Y, Chen N, Mao S J, et al. LSTM based encoder-decoder for short-term predictions of gas concentration using multi-sensor fusion. Process Safety and Environmental Protection, 2020, 137: 93-105.
［18］Chung J, Gulcehre C, Cho K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. 2014-12-11.https://arxiv.org/abs/1412.3555.
基于AR_CLSTM的多元时间序列预测分析
乔钢柱， 宿荣， 张宏飞
（中北大学 大数据学院， 山西 太原 030051）
摘要:时间序列是一种广泛应用于电量预测、汇率预测、太阳能发电量预测等各种领域的数据， 预测其变化具有重要的意义。 与LSTM相结合的编码器-解码器被广泛应用于多元时间序列预测。 由于编码器只能将信息编码成固定长度的向量， 因此模型的性能随着输入序列或输出序列长度的增加而迅速下降。 为此， 提出了基于编解码结构与线性回归的组合模型（AR_CLSTM）， 该模型使用基于时间步的注意力机制使解码器能够自适应选择过去的隐藏状态并提取有用的信息， 并利用卷积的结构学习多元时间序列不同维度之间的内在联系， 同时结合了传统的线性自回归方法来学习时间序列的线性关系， 从而实现在编解码结构上进一步降低时间序列预测的误差， 改善多元时间序列的预测效果。 实验结果表明， AR_CLSTM模型在不同的时间序列预测上表现良好， 其均方根误差、均方误差、平均绝对误差均下降显著。
关键词:编解码； 注意力机制； 卷积； 自回归模型； 多元时间序列
引用格式：QIAO Gangzhu， SU Rong， ZHANG Hongfei. Multivariate time series prediction based on AR_CLSTM. Journal of Measurement Science and Instrumentation, 2021, 12（3）： 322-330. DOI： 10．3969／j．issn．1674-8042．2021．03．010
[full text view]
