EdgeLSTM: Towards Deep and Sequential Edge Computing for IoT Applications

Index

Journal:: “IEEE/ACM Transactions on Networking” url:: https://ieeexplore.ieee.org/abstract/document/9422203

Abstract

The time series data generated by massive sensors in Internet of Things (IoT) is extremely dynamic, heterogeneous, large scale and time-dependent. It poses great challenges (e.g. accuracy, reliability, stability) on the real-time analysis and decision making for different IoT applications. In this paper, we design, implement and evaluate EdgeLSTM, a unified data-driven system to enhance IoT computing at the network edge. The EdgeLSTM leverages the grid long short-term memory (Grid LSTM) to provide an agile solution for both deep and sequential computation, therefore can address important features such as large-scale, variety, time dependency and real time in IoT data. Our system exploits the advantages of Grid LSTM network and extends it with a multiclass support vector machine by rigorous regularization and optimization approaches, which not only has strong prediction capability of time series data, but also achieves fine-grained multiple classification through the predictive error. We deploy the EdgeLSTM into four IoT applications, including data prediction, anomaly detection, network maintenance and mobility management by extensive experiments. Our evaluation results of real-world time series data with different short-term and long-term time dependency from these typical IoT applications show that our EdgeLSTM system can guarantee robust performance in IoT computing.

Top Notes

Quick Reference

our EdgeLSTM system uses the Grid LSTM neural network to predict the trend of time series data. In order to prevent the model from over-fitting and improve the generalization performance, several regularization techniques have been adopted to train the model. Then, the error data set is constructed according to the difference between the predicted data values and the true data values, and later used to train a multiclass support vector machine (SVM) [17] to perform data classification. Our EdgeLSTM system can realize the full potential of edge computing to support the data-driven IoT applications.

The EdgeLSTM Algorithm and Its Application: The key phases of EdgeLSTM as a unified system to process and analyze IoT time-series data as shown in Fig. 2 is described in Algorithm 1. Given q samples after data processing, each sample is a time series data (x1,⋯,xt,⋯,xT) , where t represents time steps in the sequence and T is the sequence length. The Grid LSTM cell can process these input samples and derive corresponding memory and hidden vectors both in time and depth dimension, and then generate their outputs to the next time step for prediction. After going through all the time steps in the Grid LSTM network, EdgeLSTM generates the hypotheses y^ of the target y as the prediction result. We use the least squares loss function loss(y^,y)=1n∑qi=1(y(i)−y^(i))2 as the cost function for our Grid LSTM model, where y(i) stands for the predicted result of the i -th sample. Subsequently, we use the Back Propagation Through Time (BPTT) algorithm to train the model. Algorithm 1 also describes the use of multiclass SVM in EdgeLSTM to process the error data set generated by its prediction model, and its application in anomaly detection.

Classical Time Series Methods. Traditional techniques follow basic principles by detecting continuous changes in the sequence and leveraging past data to predict the development of trends, using a statistical indicator such as cumulative sum (CUSUM) and exponentially weighted moving average (EWMA) in a time window to detect potential changes in the distribution. The length of the time window usually needs to be predetermined. A representative model of time series method is the autoregressive moving average (ARIMA) [26]. The ARIMA model transforms non-stationary time series into stationary time series and predicts future values from past and present values of time series. However, it has low accuracy and only suitable for the short-term prediction. A simple feed-forward neural network was proposed in time series data processing [47]. It predefines sliding windows to build features to make use of the relation between time series, but this approach has too much parameter dependency and may cause instability. In addition, there are some sequence models dealing with sequential data, such as conditional random field model [48], Kalman filter [49], Markov model [50], which are lacking the ability to learn long-term dependency.

Tasks

Annotations