# Stock Price Forecasting with LSTM

## What is LSTM Network

Long short-term memory (LSTM) network is a type of recurrent neural network (RNN). RNN is a type of artificial neural network that uses sequential data or time series data. There is a time relationship (memory) between samples in RNN, unlike in the feed-forward neural network.

<figure><img src="https://4146991827-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FIsb2SAYKLkGlVOGOY0EE%2Fuploads%2FnUKtVXWege9o9S3sHDCE%2Frnn_vs_feed_forward.png?alt=media&#x26;token=d7056235-b0dd-4b88-993d-3d39a5ce6151" alt=""><figcaption><p>Unlike a feed-forward network, RNN has memory represented by a loop in the neuron. Source: <a href="https://medium.com/@shekhawatsamvardhan/types-of-neural-networks-recurrent-neural-networks-7c43bd73e033">https://medium.com/@shekhawatsamvardhan/types-of-neural-networks-recurrent-neural-networks-7c43bd73e033</a></p></figcaption></figure>

LSTM are an upgraded variant of RNN. RNNs suffer from vanishing and exploding gradient problems. The vanishing and exploding gradient problems occur when the gradients become either too small or too large during backpropagation.

LSTM can refer to:

* An **LSTM unit** (or neuron),
* An **LSTM layer** (many LSTM units), or
* An **LSTM neural network** (a neural network with LSTM units or layers).

<figure><img src="https://4146991827-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FIsb2SAYKLkGlVOGOY0EE%2Fuploads%2FrRdq9HvqCBRLLXZnOQoE%2Flstm_layer_and_neuron.png?alt=media&#x26;token=ee673963-344d-4ab3-9422-3e61e2ca31f7" alt=""><figcaption><p>Illustration of an LSTM layer and unit (or neuron) are, and the rightmost image shows the internal structure of a single LSTM unit. Source: <a href="https://stats.stackexchange.com/questions/365428/difference-between-a-single-unit-lstm-and-3-unit-lstm-neural-network">https://stats.stackexchange.com/questions/365428/difference-between-a-single-unit-lstm-and-3-unit-lstm-neural-network</a></p></figcaption></figure>

To implement LSTM using Keras, this is the basic code:

```python
model = Sequential()
model.add(LSTM(32, input_shape=(10, 64)))
```

The architecture has 32 neurons. The 10 represents the timestep value. The 64 represents the number of features.

## LSTM Model for Stock Price Forecasting

In this section, I explain the very basic LSTM model for stock forecasting. It only uses price as a feature, and the number of timesteps is one. You can get the Jupyter notebook file from my repository.

{% embed url="https://github.com/weenslab/coding_for_finance/blob/main/lstm_stock_forecast_0.ipynb" %}

### Import Library

To begin our project, we need to install the `yfinance` library to get stock price data. Then, we need to import several libraries.

{% embed url="https://gist.github.com/weenslab/183618486db5000e93a8e3bb679e3cbf" %}

### Download Dataset

I use the stock price of BBCA, one of the four big banks in Indonesia. To download the data set, you can specify the start date of the data along with the time zone. The timeframe of stock price data is 1 day.

{% embed url="https://gist.github.com/weenslab/b28f60c5c70808bfc26155a0c7acbc95" %}

### Remove Unnecessary Columns <a href="#remove-unnecessary-columns" id="remove-unnecessary-columns"></a>

We want to do time series forecasting with only one feature, which is price. Therefore, we need to remove other features from the dataset.

{% embed url="https://gist.github.com/weenslab/9ff06056f9a1d1b30b0b9cc1c84853db" %}

### Data Normalization

Normalization converts the values of numeric columns in the dataset to a common scale, which improves the performance of our model. To scale the training dataset, we utilize Scikit-Learn's `MinMaxScaler` with numbers ranging from zero to one.

{% embed url="https://gist.github.com/weenslab/411eb945ef459fb76a02ca2ce1833239" %}

### Incorporate Timesteps into Data <a href="#incorporate-timesteps-into-data" id="incorporate-timesteps-into-data"></a>

Before LSTM can be used, the time series dataset must be reframed as a supervised learning dataset. From a sequence to pairs of input and output sequences. This function, `series_to_supervised()`, is used to convert the data.

{% embed url="https://gist.github.com/weenslab/9b6b3df5201b34f1f82130e57cfee130" %}

Technically, in time series forecasting terminology the current time $$(t)$$ and future times $$(t+1, t+n)$$ are forecast times and past observations $$(t-1, t-n)$$ are used to make forecasts.

### Train-Test Split <a href="#train-test-split" id="train-test-split"></a>

Here, we divide the data into a training set and a test set.

{% embed url="https://gist.github.com/weenslab/24697df45b49c87cbe3e8c801e0e3e92" %}

### Convert into 3D Arrays <a href="#convert-into-3d-arrays" id="convert-into-3d-arrays"></a>

LSTM takes as input 3 dimension tensors (`batch`, `timesteps`, `feature`). So, we need to convert the data into 3D arrays.

{% embed url="https://gist.github.com/weenslab/0dd9dcce0c6a1c68eb68caf494a8995b" %}

### Create LSTM Model <a href="#create-lstm-model" id="create-lstm-model"></a>

The LSTM layer is added with the following arguments:

* The 150 units is the number of LSTM neurons and also the dimensionality of the output space.
* The `input_shape` is the shape of the training dataset (`timesteps`, `feature`)
* The `return_sequences=True` is necessary for stacking LSTM layers so the consequent LSTM layer has a three-dimensional sequence input.

We add `Dropout` layer with 0.3 rate. It means that 30% of the layers will be dropped. Finally, we add the `Dense` layer that specifies an output of one unit.

{% embed url="https://gist.github.com/weenslab/ad897d6d5b36554579dec54b420af696" %}

### Training

We train the model to run for 20 epochs with a batch size of 32. The epochs are the number of times the learning algorithm will work through the entire training set. The batch size is a number of samples processed before the model is updated.

{% embed url="https://gist.github.com/weenslab/f2d0770b34d69ca9b8fc03a06f85a679" %}

### Prediction

We make prediction with the test set.

{% embed url="https://gist.github.com/weenslab/0f8cb17f286c27942c5eee78fc14c162" %}

### Inverse Normalization <a href="#inverse-normalization" id="inverse-normalization"></a>

The result is still in normalized value, so we have to invert back to normal value.

{% embed url="https://gist.github.com/weenslab/4da925264093f6c240766cc05c054c20" %}

### Comparison with Ground Truth <a href="#comparison-with-ground-truth" id="comparison-with-ground-truth"></a>

We compare our model prediction result with the ground truth. We calculate the root mean squared error between the true price and the predicted price.

{% embed url="https://gist.github.com/weenslab/eda38da81fcd5cddac48ee9fddeaa930" %}

If you see the comparison between the predicted price and the true price, it seems that the model does a pretty good job. But if you look closely, the predicted price is just a delayed version of the true data. Therefore, the model is not yet useful for real-life trading. The predicted price is just like a moving average indicator with a very small window.

## References

* Types of neural networks: Recurrent Neural Networks, [https://medium.com/@shekhawatsamvardhan/types-of-neural-networks-recurrent-neural-networks-7c43bd73e033](https://medium.com/@shekhawatsamvardhan/types-of-neural-networks-recurrent-neural-networks-7c43bd73e033)
* What is the difference between LSTM and RNN?, [https://ai.stackexchange.com/questions/18198/what-is-the-difference-between-lstm-and-rnn](https://ai.stackexchange.com/questions/18198/what-is-the-difference-between-lstm-and-rnn)
* Difference between a single unit LSTM and 3-unit LSTM neural network, [https://stats.stackexchange.com/questions/365428/difference-between-a-single-unit-lstm-and-3-unit-lstm-neural-network](https://stats.stackexchange.com/questions/365428/difference-between-a-single-unit-lstm-and-3-unit-lstm-neural-network)
* What is the architecture behind the Keras LSTM Layer implementation?, [https://stackoverflow.com/questions/49892528/what-is-the-architecture-behind-the-keras-lstm-layer-implementation](https://stackoverflow.com/questions/49892528/what-is-the-architecture-behind-the-keras-lstm-layer-implementation)
* How to Convert a Time Series to a Supervised Learning Problem in Python, [https://machinelearningmastery.com/convert-time-series-supervised-learning-problem-python/](https://machinelearningmastery.com/convert-time-series-supervised-learning-problem-python/)
