32 Introduction to Stationary Time Series

Dr. Harmanpreet Singh Kapoor

Module 38: Introduction to Stationary Time Series

  • Learning Objectives
  • Introduction.
  • Time Series as Stochastic Processes
  • Stationary Time Series
  • Definitions
  • Types of Stochastic Model
  • Stationarity Testing
  • Software awareness vs. Time Series Analysis
  • Summary
  • Suggested Readings

 

  1. Learning Objectives

 

The aim of this module is to give a basic introduction about stationary and non-stationary time series and understand its concepts and significance. We will discuss its properties as well as try to explain its importance in other areas. In this modules, examples are added to help students in understanding the topic in an easy manner.

 

  1. Introduction

 

Time series analysis has wide application in the practical world. Then the questions that comes to the mind are:

 

What is the purpose of using this tool?

Why time series analysis is considered as the most important tool in the financial field?

How this analytical tool is considered as more useful than any other methods?

Many of these types of questions will be in your mind. We will try our best to answer all these types of questions in this model.

We studied about the time series and how to measure its components in the module “Introduction to Time Series and its importance”. We are rewriting here again about time series.

What is time series? Time series is the sequence of observations that record the data at regular time intervals such as: retail price index.

The main purpose of using this analysis is for:

the description of the data.

the construction of the models which fits the data. Forecasting future value

 

Let’s say a business analysts want to analyze the future situation of the company. First step is to analyze the previous year’s figures so that using those figures they will be able to comment on it. Here the analysis has been done with time-dependent data. Another example is that predicted weather predates the monsoon prediction in the year 2018, for this, he/ she has used the analysis of previously recorded data and has concluded about the status of monsoon.

 

In the most practical situation, observation data relies on time. In these situations, time series analysis is more useful. Linear regression analysis is also used in forecasting but it is less useful because it considers the relationship of one variable with other variables such as hours spent in a day for study correlated with the marks score or yield of crop depends on other factor etc. The main thing in regression analysis is that here observation values may not be observed with time but in time series observations are observed with time. Therefore it is important to understand the relationship of variable over time.

 

For example, the price of share increase or decrease with time. Here the weather condition on a particular day depends on its previous day weather. Hence it is important that one should study the relationship of a variable with its previous values through modeling that not only helps in the fitting of the data in more appropriate manner but also for future predictions.

 

Time series analysis is used to understand the behavior of any variable over time. Time series analysis consists of different techniques, theory and methods to predict the future value based on previous observed value. This prediction theory is therefore classified as Time series forecasting.

 

This analysis is used from ancient time period to present time period. In ancient time, people has used their own knowledge to predict future value such as weather prediction by looking at the weather condition of the sky.

 

Time series analysis and forecasting has wide applications in different areas. Time series analysis is mostly used in business and economic area. Some other applications areas are: Weather Forecasting, Pattern Recognition, Earthquake Prediction, Mathematical Finance etc.

 

  1. Time Series as Stochastic Process

 

A Time series is a stochastic process because time series is a sequence of observations and sequence of observations is basically a realization of sequence of ordered random variables i.e. stochastic processes.

A brief about stochastic processes is given here so that one can understand it.

 

Definition: “Stochastic process is families of random variables which are function of, say, time. A stochastic process is also known as random process or random function.”

It is denoted as: {  |     ≥ 1}; where                             is a random variable at time n.

 

Basically it is a sequence of random variables that model the value of process at time n

Definition: “A time series process is a stochastic process indexed in discrete time and continuous state space”.

 

For example: daily closing price of share market.

Such series can be written as:{  |     = 1,2,3, … . }

 

  1. Stationary Time Series

 

In general terms “stationary means stability”. In time series analysis stationary means stability of the stochastic process with respect to its random variable. It means, there is no changes in the time series with respect to changes in the time interval, this is known as Stationary Time series.

 

In literature, there are two types of stationary. These are

 

  • (a) Strictly Stationary
  • (b) Weakly Stationary

 

 

4.1 Strictly Stationary

 

Definition:

 

“A  stochastic  process  is  said  to  be  Stationary  or  Strictly Stationary  Times,  if  joint  distribution

of      1 ,  2 , … . . ,     and  + 1 ,  + 2,……..,  + are identical for all    1, … …    + 1, … … .     +

in j and all integers n”.

 

This means that all statistical properties remain constant as time elapses i.e. mean, variance and covariance of the process remain constant. A white noise series is stationary series with mean 0 and variance 2.

 

Hence one can observe that the mean and covariance values are constant. So given series is said to be a stationary time series.

 

Also it is difficult that all the time series must satisfy all the conditions of the strictly stationary. For example, it is difficult in general that variance remain constant over time. So in general, second type of stationary is mostly used due to its flexible nature.

 

4.2. Weakly Stationary Time series

 

A stationary or strict stationary condition does not hold in real life because it is not necessary that all properties are constant with to time. So to overcome the problems of strictly stationary, another type “Weakly Stationary Time Series” is introduced.

 

Definition:

 

“A Weakly Stationary Time series requires that mean and variance of the process remains unchanged as time elapses but covariance depend on its time difference i.e. lag.”

 

Such that

(  ,                  ) =   [(  ,   (  ))(  ,   (  ))]

 

where     ,        are the random variable that model the value of process at time    and    respectively.

This covariance depends on lag or time difference i.e. (  −  ) if                                                                        > .

Now in the following section, we will discuss some properties of time series that help in modeling the data. These properties help in deciding the model to fit the data.

 

Example:

 

Let be the independently and identically distributed standard normal random variable. Show whether the following process is Stationary time series or not? Also check that it satisfies the properties of weakly stationary process.

=   −1 +

Solution: For stationary process mean and covariance should be same.

 

Time series has constant mean and covariance change as time difference change.

 

 

This covariance value at lag 0 is 2. Now, find covariance value for 1 lag, it is 1. Covariance value for lag greater than 2 will be 0. So, one can see that as time interval changes covariance value changes i.e. covariance depends on the lag or time difference. Thus, this process is the weakly stationary time series.

 

Here, we can see that the mean is zero and covariance term is not same for all lag basically it is a function of time lag. Hence we can say that the process is not strictly stationary but it is weakly stationary.

 

  1. Definitions
  • (a) Autocovariance function: The covariance between any pair of elements of same variable of a stationary sequence, depends on its lag difference, this covariance is known as autocovariance.

In the stationary process the means are constant and covariance are function of time lag. Therefore the covariance between two variables remain same have same lag i.e time difference. Hence in the autocovariance function, there is no need to mention about the starting time but it is necessary to mention time lag .

 

On the other hand, if a process is non-stationary then the Autocovariance function depend on two variable, first is time and second one is lag .

 

It means that covariance of variable changes with time although having same lag.

Then the notation for the autocovariance is denoted as:

 

 

The ACF of a purely indeterministic process satisfies → 0 → ∞. It means that as we increase the time lag between variable then the autocorrelation between the variables converge to zero i.e. no relationship.

 

If the process is not stationary then we have the different notation for autocorrelation like autocovariance.

 

The autocorrelation function for a non-stationary series defined

 

  • (e) Parsimony principle: While building a proper time series model we have to consider the principle of parsimony. According to this principle, always consider the model with smallest possible number of parameters to be selected for representing an adequate time series data. Out of most suitable models, select simplest one because complicated model has large number of parameters, large number of parameter shows overfitting of model, overfitting is not suitable for future forecasting. So for future forecasting of time series model, we select most parsimonious model among all the possibilities.
  1. Types of Stochastic Model

 

Basic introductory time series model “Multiplicative and Additive model” contain error term in terms of irregular component but this component is not random. Error term must be considered as a random quantity because variations in real life problems are random. Thus by adding an error term in the time series model, the basic model of time series is turned into stochastic time series model. The selection of a proper model is extremely important as it reflects the underlying structure of the series and this fitted model in turn is used for future forecasting. These stochastic models are based on the assumption that corresponding time series must be stationary time series.

 

In general time series data can have many forms. Different stochastic process models are used to fit the data. These are some linear time series model that is mostly used in literature. These models can only be used with stationary time series. Such as:

Autoregressive model (AR) :

 

Autoregressive model explains the current value of a random variable say as a linear combination of its past values with additionally externally generated random variation. For example: current price of a commodity is based on the previous prices of the commodity.

 

Graphical method: in this method, graph plotted between given data set of daily total female births in California and every 12 months.

This graph shows random sequence with constant mean and variance. Thus, this process or time series is Stationary time series.

 

Correlogram method: Correlogram method use ACF and PACF values of the given time series.

 

If the ACF values are exponentially decreasing and PACF values rapidly goes to 0, this Time series is Stationary time series and fit Autoregressive model. Parameter of the AR model can be defined by the PACF values. After that lag value (parameter value), all PACF values goes to 0 that PACF value is the parameter of the AR model.

 

If ACF values rapidly goes to 0 and PACF values exponentially decreasing, then this time series is Stationary time series and fit Moving average model. Moving average model parameter defines according to ACF value.

 

Correlogram is:

 

This correleogram shows ACF values exponentially decreasing and PACF values rapidly goes to 0 after 2 PACF values. So this is a AR model with parameter 2 that is daily total birth of female in California,1959 time series is stationary time series and fit the AR(2) Stochastic time series model.

 

  1. Software Awareness vs Time Series Analysis

Time series analysis requires computer awareness because it involves long and hard mathematical calculations. These type of calculations cannot be solved by hand for a long time. There are some statistical software available to solve this calculation.

 

Some of these are: Eviews, R, SPSS, SAS, Python, Excel

The most useful software is Eviews and R.

 

R is user-friendly and free software available. It is very easy to use this software, there are lots of content available on the internet.

 

  1. Summary

 

These Stochastic Time series model used to prediction and analysis of a Time series. These models are appropriate only when Time Series is Stationary Time series. In case of Non-Stationary Time series, there are other methods available in the literature. Stochastic Time series models for Non-Stationary Time series are also appropriate when given Time series is Stationary Time series. For that convert Non-Stationary Time series into Stationary Time series.

 

Methods to convert Non-Stationary into Stationary Time series and Stochastic Time series models for Non-Stationary time series will be discussed in the next module.

 

  1. Suggested Readings

 

  • Gupta, S. C. and Kapoor, V. K., Fundamentals of Applied Statistics, Sultan Chand & Sons, New Delhi, 2009.
  • Gupta, S. P., Statistical Methods, Sultan Chand & Sons, New Delhi, 2012.
  • Gupta, S.C., Fundamental of Statistics, Himalaya Publishing House, Nagpur, 2016.
  • Sharma, J. K., Business Statistics, Vikas Publishing House, 2014.
  • Tsay, R. S., Time Series and Forecasting: Brief History and Future Research, Journal of the American
  • Statistical Association, Vol.95, pp. 638-643, 2000.
  • Agresti, A. and B. Finlay, Statistical Methods for the Social Science, 3rd Edition, Prentice Hall, 1997.
  • Daniel, W. W. and C. L. Cross, Biostatistics: A Foundation for Analysis in the Health Sciences, 10th Edition, John Wiley & Sons, 2013.
  • Gupta, S. C. and V. K. Kapoor, Fundamentals of Applied Statistics, Sultan Chand & Sons, New Delhi, 2009.
  • Gupta, S. P., Statistical Methods, Sultan Chand & Sons, New Delhi, 2012.
  • Hogg, R. V., J. Mckean and A. Craig, Introduction to Mathematical Statistics, Macmillan Pub. Co. Inc., 1978.
  • Meyer, P. L., Introductory Probability and Statistical Applications, Oxford & IBH Pub, 1975.
  • Sharma, J. K., Business Statistics, Vikas Publishing House, 2014.
  • Triola, M. F., Elementary Statistics, 13th  Edition, Pearson, 2017.
  • Tsay, R. S., Time Series and Forecasting: Brief History and Future Research, Journal of the American Statistical Association, Vol.95, pp. 638-643, 2000.
  • Weiss, N. A., Introductory Statistics, 10th Edition, Pearson, 2017.