33 Introduction to Non-Stationary Time Series

Dr. Harmanpreet Singh Kapoor

Module 39: Introduction to Non-Stationary Time Series

  • Learning Objectives
  • Introduction.
  • Behavior of Non-Stationary Time Series
  • Types of Non-Stationary Processes
  • Techniques to remove Non-Stationarity
  • Stochastic Time Series Process
  • Summary
  • Suggested Readings

 

  1. Learning Objectives

 

The objective of this module is to give an introduction to non-stationary time series. After that the behavior of non-stationary time series will be discussed. Types of non-stationary process like random walk without drift, random walk with drift etc will be discussed. Different techniques will be discussed on how to remove non-stationary factors from the series.. We will also discuss some stochastic models and how to use stochastic models for non-stationary time series and its application part. This module will be helpful to understand non-stationarity conditions with examples in a very simple manner. The steps will be discussed in detail with example about how to choose the best model that best fits the data.

 

  1. Introduction

 

In practical cases, the data of time series are available in the raw form that means one cannot apply the time series tools and techniques on them in direct manner. In most of the cases, data of time series are far away from stationary. In original time series, there are many types of fluctuations available like variability in mean, variability in variance, volatility and relation with other variable(s). Forecasting of respective time series cannot be done in the presence of these fluctuation. If forecasting will be done with fluctuations, mean variance of the forecasted values will fluctuate. Thus stationary time series is considered as standard time series for forecasting because mean and variance value of the forecast values remain same as the original time series that is there is no change or significant difference on the mean and variance values as time passes. One can use this stationary series to make better forecasting of the series. Hence stationary process is the essential condition for accurate forecasting. If time series is non-stationary, then one need to remove non-stationarity factors from the series and convert it into stationary process.

 

Let us suppose a case of asset prices in the share or property market. As asset prices are recorded at a particular time period. Recorded prices fluctuate over time which means there is no stability in various statistical characteristics of the object like mean and variance. If we are interested in the prediction of the future asset value. Then predicted future asset value will not be accurate and reliable due to instability in the characteristics of the prices that is measures through statistical properties such as mean, variance and covariance of the recorded data. It means statistical properties are not constant, it varies over time.

From previous module, “Introduction to Stationary Time series”, if statistical properties are constant over time, respective data are known as stationary data and the process is known as stationary process.

 

If stationarity condition doesn’t hold either strong or weak, then the process is considered as a non-Stationary. One can also use the term non-stationarity for this type of the process

 

Definition

 

“If the statistical properties of the recorded data are changing over time that means mean and variance are not constant, recorded data is known as non-stationary data and process is known as non-stationary process or non-stationary time series.”

 

This type of non-stationarity condition is mostly seen in financial and economics field as financial market shows either increasing and decreasing trend or fluctuations in the market. This is among the best example to understand non-stationarity. Now let us define the behavior of the non-stationary time series.

 

  1. Behavior of Non-Stationary Time series

In this section some most important key features will be discussed about behavior of non-stationary.

Such as:

  • Ø If the graphical representation of the recorded data shows upward or downward trend, cycles, random walks with/without combination of trend, cycle and random walks, then time series is non-stationary data and have non-stationarity condition.

In the following figure, one can see the different components and their combinations that exits among different time series through different colors for better understanding.

 

The terms like random walk etc. will be discussed further for better understanding of the concept in the module.

  • Ø Non-Stationary data possess non-stability into the mean and variance in long run. This means that variance goes to infinity as time goes infinite and mean value never become stable on one value as time passes.
  • Ø Non-Stationary data can’t be predicted, modeled and forecasted due to presence of non-stability factors that lead to variation in the values of statistical properties like mean, variance over time. To get consistent and reliable outcome of non-stationary data, there is a need to transform non-stationary data into stationary data. Under stationarity condition forecasted value will be consistent and reliable.

 

  1. Types of Non-Stationary processes

 

Before discussing the transformation of the non-stationary time series into stationary time series, different types of non-stationary process will be discussed. These non-stationary process may be helpful in the transformation of the non-stationary process.

 

Some well known non-stationary processes are:

  • Ø Random walk without drift
  • Ø Random walk with drift
  • Ø Deterministic trend

4.1 Random walk

 

In simple term random walk is a sequence of random changes, either in the variable or process, subject to condition. Random walk is also defined as a path taken by a point and quantity that moves in steps and direction of the each step will be decided randomly.

 

Suppose you want to predict chances of accident of a drunk who is walking on road. One truck is coming towards him and he is walking randomly. He is in the middle of the road now and truck is coming towards him. He takes steps randomly left, right, forward or backward from the current position. You can’t predict if that person will meet with an accident because he is taking steps randomly in random direction.

 

Drunk person taking moves step by step randomly so this process is known as random walk.

Now let us discuss statistical definition of the random walk in statistical terms.

 

“Random walk is a stochastic process consisting of a sequence of changes in stepwise manner each of which step characteristic is determined by chance or randomly”.

 

Random walk theory is mostly used in stock prices and exchange rates. Random walk is discussed in two ways in the literature, one is random walk without drift and other is random walk with drift.

 

Random walk without drift: Random walk without drift; simply is a random walk in which today’s stock price will be equal to yesterday’s stock price plus a random stock. This is a non-mean reverting process which means that it can move towards positive and negative side and variance changes as time changes in long run, so that it can’t be predicted.

 

Here the price of the stock at time t will depend on the price of the stock at time − 1 and the random movement of the stock price at time t.

 

Random walk with drift: Random walk with drift, tomorrow’s value will be equal to the today’s value plus drift plus random variable. Drift is the constant and average value of measuring changes. It can be used to measure trend, cycle. It is not accurately predictable because there is no constant mean and variance.

 

Mathematical expression for the random walk with drift:

Here the price of the stock at time t will depend on the price of the stock at time − 1 and the random movement of the stock price at time t.

 

Random walk with drift: Random walk with drift, tomorrow’s value will be equal to the today’s value plus drift plus random variable. Drift is the constant and average value of measuring changes. It can be used to measure trend, cycle. It is not accurately predictable because there is no constant mean and variance.

Mathematical expression for the random walk with drift:

 

  1. Techniques to Remove Non-Stationarity

 

Non-Stationary means lack of stationarity in other words the absence of stationarity or stability of statistical properties in the data. Lack of Stationarity is caused by the presence of non-stationary factors like deterministic trend, deterministic cycle etc. It is important to detect these factors at the initial stage so that after removing these factors one can apply statistical methods to forecast prospective values for the original time series.

 

This section will introduce the techniques for detecting non-stationarity factor. There are many techniques available in the literature of time series analysis.

 

Some of them are discussed in the module. One can refer to the literature for in-depth knowledge of these methods.

 

5.1. Least Square Method for Elimination of Trend

 

Least square trend method is used to measure trend component. Basically the purpose of measurement of component technique is to first identify the components, then isolate it and eliminate from the data if possible. As some of the components like irregular variation or random term is difficult to eliminate completely from the data or series. There are many method like moving average method, method of semi average that are available in the literature but least square method is the best method among the all methods to eliminate trend. Actually, a time series cannot remove overall trend. A time series will always keep some trend value. But least square method removes trend as possible as it can. Least square method is the simplest and widely used method because of its unbiasedness. Detail procedure of this method is described in the module “Measurement of Secular Trend”. One can refer this module for mathematical formulation of the method. Least square method removes all type of trend like linear trend, quadratic trend, curvilinear trend and exponential trend etc.

 

5.2. Differencing / Unit root test

 

Before future forecasting of a time series, respective time series should be stationary time series. Since stationary time series have constant mean and variance in the long run. Forecasting is very important feature of a time series. Most of the time series show variability in original data. They are not applicable to forecast future value. There is a need to transform non-stationary data into stationary data. Merely stationary time series provides better and consistent result to any recorded data due to stability prevails in the statistical properties. In literature, there are many methods available to transform the non-stationary process into stationary process. But differencing is very important method for transformation. Differencing method is also known as unit root test.

 

“Differencing is a method to detrend the recorded data. In this method difference of variables are taken. This difference is taken on the basis of the characteristic of the recorded data.”

Suppose there is a random process     , the first difference of a series       is given as:

∆   =              −   −1

 

If, after taking first differencing, the process will fulfill the requirements for the stationarity. Then the transformed process derived from the original process is stationary time series and original process or time series is called integrated to order one, and denoted by I(1).

 

If the first differencing does not fulfill the requirements of the stationarity, then there is a need to take second differencing by the equation:

∆∆   = ∆2                      = ∆   − ∆    −1

 

If the second difference follows stationarity properties, then second difference is stationary but not the original time series. Then the original series is considered as integrated of order two and denoted as I (2).

 

If a process differenced d times in order to induce stationarity, the series is called integrated of order d and denoted by I (d).

 

One can see that still process is non-stationary but their differences is stationary.

 

5.3. Seasonal differencing:

 

If time series is affected by seasonal factors and man-made factor, time series has seasonal variation. Seasonal variation shows short term movement in the time series like daily, hourly, weekly, monthly, quarterly but less than a year. There are many methods discussed in the module “Measurement of Seasonal, Cyclic and Irregular Variation” to measure seasonal, cyclic and irregular components. There is one more method to eliminate seasonal variation from the respective time series. This method is known as seasonal differencing. Seasonal differencing is based on the season factor. If time series is affected by monthly seasonal factors, seasonal differencing will be taken on 12 period. In that case differencing is done in fractions rather than integers. Similarly, this can happen for other season factors.

 

Now we will discuss a case where monthly seasonal differencing are taken.

For example:

Let us consider that the time series records the monthly average temperature in India. A model is of the form:

where    is the constant parameter value;

 

is the periodic function with period 12;

 

is a stationary time series.

 

 

One can see that this seasonal difference is stationary. So one can say that after seasonal difference, series can be reconstructed. Now after taking the seasonal difference the transformed series is stationary.

 

5.4. Method of moving average:

 

Method of moving average is used to measure trend component. Moving average method are also used to eliminate periodic variation from the time series. Method of moving average have some loss of information from the given time series. This method is very simple and based on the some period. This period decide on the basis of time series movement. Number of cycles defines the period of moving average period. Moving average method is discussed in “Measurement of Secular Trend” in detail. One can refer these modules for detail learning.

 

  1. Stochastic time series process:

 

After addressing a non-stationary time series to stationary time series, next step is to fit a model on the respective time series. Many time series stochastic models are used to fit a transform time series. Most widely used and important stochastic models are discussed here. These are Auto-Regressive Integrated Moving Average and Integrated Moving Average model.

 

6.1. Auto-Regressive Integrated Moving Average Process (ARIMA)

 

An autoregressive integrated moving average process can be denoted as ARIMA (p,d,q). When time series is non-stationary, we can’t directly apply ARMA (p, q) of that series. Thus first we have to convert non-stationary series to stationary series by taking differencing.

 

 

After differencing d times we will get series that fulfill the requirement of stationarity then we can apply ARMA (p, q) model on the differenced series and the original model is known as ARIMA (p, d, q) model.

where p is the number of lags of the dependent variable (AR terms).

d is the number of differences required to take in order to make the series is stationary.

q is the number of lags of the error term (MA terms).

 

6.2. Integrated Moving Average Model

 

An integrated moving average model can be represented as IMA (d,q). An integrated moving average is simply an ARIMA model with p =0. That is, the IMA (d,q) model is the same as the ARIMA (0, d, q).

 

The IMA(d,q) is a moving average which has been integrated d times. Here, we will study the simplest case, the IMA (1, 1), also known as ARIMA(0, 1, 1). The model can be written as:

−      −1 =     −                −1

where: a lies between −1 and 1 (because of the invariability condition).

Since      = 1, the series {Zt} is non-stationary.

 

Example

 

Let us consider an example of quarterly data for the consumer price index (CPI) and gross domestic product (GDP) of the UK Company. Here, this time series will be used to check non-stationarity. If it is non-stationary then convert it into stationarity, and to find appropriate stochastic model.

 

Solution: – ( Here, we are using Eviews software to solve this example)

To check non-stationarity in the time series, steps are given below:

To see the characteristic of the time series, the first step is to plot the data. One can see from Figure 2 (first figure top left) that represents the plot of the data. This shows a deterministic trend in the data. So, this time series is non-stationary Time series.

 

The second step is to transform the non-stationary into stationary time series by differencing and plot it. Figure 2 (second figure top right) shows the plot of differencing time series. From this figure, one can see that difference time series is stationary time series.

 

One can take log to smooth the time series. Log time series shows non-stationary and difference of it is stationary time series.

 

In Figures 4 (a)-4 (e), coefficient column shows parameters value like α, β, θ for all ARMA. ARMA model are discussed in detail with mathematical equations in the module “Introductions to Stationary Time Series”.

 

Probability column shows whether parameter is significant or not. Parameter is significant if probability values is less than 0.05 otherwise not. Here, significant means that parametric value is acceptable. Adjusted R- square means how much data is fitted by the model. In other words, how much model is successful in explaining the variation of the data? As the value of Adjusted R-square is close to one Result of all possible ARMA model

 

Table 1 shows the combine result of all possible ARMA model. From all these model, we have to choose one of the best and appropriate model.

 

The term degree of freedom is used in the table. It means independent observations that are included in the respective model.

 

From the Table 1, from second row to seventh row, we have values of t-statistics that are used to test the significance of the parameters. ARMA (1, 5) and ARMA (1, 4) are the most appropriate model among other model, because their adjusted R squared value is greater than all other possible time series model i.e. 0.218 & 0.23.

 

Since, ARMA (1, 5) is not appropriate model because MA (5) is not significant. MA (5) is not significant because it is less than 0.05. One can check this by fitting the MA (5) model on the data using the Eviews software. Thus, ARMA (1, 4) is the most appropriate one as:

 

It has Adj. 2is higher than all model It satisfies the stationary condition So, one can conclude that ARMA (1, 4) is the parsimonious model for GDP data of UK Company.

 

  1. Summary

 

By studying this module, one can get knowledge about non-stationary time series. Behavior of non-stationary time series is discussed. Some of the non-stationary models like random walk with drift, random walk without drift are discussed. Definitions and notations are also given for betterunderstanding. Examples of non-stationary Time series, techniques to remove non-stationarity factors, stochastic time series model for non-stationary time series are discussed. In this module, ARIMA models with different order are discussed and how to choose the best model is discussed. The best model is further used to forecast future values. This future forecasting can be done by using Box-Jenkins methodology. This methodology will be discussed in the module “Box-Jenkins Methodology”.

 

  1. Suggested Readings:

 

  • Agung, I. G. N., Time Series Data Analysis Using Eviews, John Wiley & Sons, Asia, 2009.
  • Box G. E. P., G. M. Jenkins, G. C. Reinsal, Time Series Analysis Forecasting & Control, 3 Edition, Prentice-Hall International, UK, 1994.
  • James, H., Time Series Analysis, Princeton University Press, 1994.
  • L ̈tkepohl, H, and M. Kr ̈tzig, Applied Time Series Econometrics, Cambridge University Press, UK, 2004.
  • Tsay, R. S., Time Series and Forecasting: Brief History and Future Research, Journal of the American Statistical Association, Vol.95, pp. 638-643, 2000.