In this post, we will cover Carter R., Griffiths W., Lim G. (2018) Principles of Econometrics, focusing on time series analysis.

Introduction

It is important to distinguish cross-sectional data and time-series data. Cross-sectional observations are typically collected by way of a random sample, and so they are uncorrelated. However, time-series observations, observed over a number of time periods, are likely to be correlated.

A second distinguishing feature of time-series data is a natural ordering to time. There is no such ordering in cross-sectional data and one could shuffle the observations. Time-series data often exhibit a dynamic relationship, in which the change in a variable now has an impact on the same or other variables in future time periods.

For example, it is common for a change in the level of an explanatory variable to have behavioral implications for other variables beyond the time period in which it occurred. The consequences of economic decisions that result in changes in economic variables can last a long time. When the income tax rate is increased, consumers have less disposable income, reducing their expenditures on goods and services, which reduces profits of suppliers, which reduces the demand for productive inputs, which reduces the profits of the input suppliers, and so on. The effect of the tax increase ripples through the economy. These effects do not occur instantaneously but are spread, or distributed, over future time periods.

Example: Plotting the Unemployment Rate and the GDP Growth Rate for the United States

Using the dataset usmacro which contain the US quarterly unemployment rate and the US quarterly growth rate for GDP. We wish to understand how series such as these evolve over time, how current values of each data series are correlated with their past values, and how one series might be related to current and past values of another.

Plotting the unemployment rate and gdp growth rate:

usmacro$date <- as.Date(
  usmacro$dateid01,
  format = "%m/%d/%Y") |>
  yearquarter()

usmacro <- as_tsibble(
  usmacro, 
  index = date
)

plot_grid(
  usmacro |>
    autoplot(u) +
    ylab("Unemployment Rate") +   
    xlab("Year") +
    theme_tq(),
  usmacro |>
    autoplot(g) +
    ylab("Growth Rate") +
    xlab("Year") +
    theme_tq(),
  ncol = 1
)

Modeling Dynamic Relationships

Time-series data are usually dynamic. What this mean is their current values are correlated to their past values and current and past values of other variables. This particular dynamic relation can be modelled using lagged values such as:

In this section, we will describe a number of the time-series models that arise from introducing lags of these kinds and explore the relationships between them.

Finite Distributed Lag Models (FDL)

Suppose that $y$ depends on current and lag value of $x$:

It is called finite distributed lag because $x$ cuts off after q lags and the impact of change in $x$ is distributed over future time periods.

Model of this kind can be used for forecasting or policy analysis.

For example in forecasting, we might be interested in using information on past interest rates to forecast future inflation. And in policy analysis, a central bank might be interested in how inflation will react now and in the future to a change in the current interest rate.

Autoregressive Models (AR)

An autoregressive model is where $y$ depends on past values of itself. An AR(p) model:

For example, an AR(2) model for the unemployment series would be:

AR models can be used to describe time paths of variables and they are generally use for forecasting.

Autoregressive Distributed Lag Models (ARDL)

A more general case which includes autoregressive model and FDL model is the ARDL(p, q) model:

For example, an ARDL(2, 1) model relating the uemployment rate $U$ to the economy $G$:

ARDL models can be used for both forecasting and policy analysis.

Infinite Distributed Lag Models (IDL)

An IDL Model is the same as FDL Model except it goes to infinity.

In order to make the model realistic, $\beta_{s}$ has to eventually decline to 0.

There are many possible lag pattern, but a popular one is the geometrically declining lag pattern:

It is possible to change an geometrically decaying IDL into an ARDL(1, 0):

It is also possible to turn an ARDL model into an IDL model. Generally, ARDL(p, q) models can be transformed into IDL models as long as the lagged coefficients eventually decline to 0.

The ARDL formulation is useful for forecasting while the IDL provides useful information for policy analysis.

Autoregressive Error Models

Another way in which lags can enter a model is through the error term:

In contrast to AR model, there is no intercept parameter as $e_{t}$ has a zero mean.

Let’s include this error term in a model:

$e_{t-1}$ can be written as:

We can then turn a AR(1) error model into an ARDL(1, 1) model:

Note that we have a constraint here which is not in a general ARDL model and nonlinear:

Autoregressive error models with more lags than one can also be transformed to special cases of ARDL models.

Summary of Dynamic Models for Stationary Time Series Data

ARDL(p, q) Model

FDL Model

IDL Model

AR(p) Model

IDL model with Geometrically Declining Lag Weights

Simple Regression with AR(1) Error

How we interpret each model depends on whether the model is used for forecasting or policy analysis. On assumption we make for all models is that the variables in the models are stationary and weakly dependent.

Autocorrelation

Consider a time series of observations, $x_{1}, \cdots, x_{T}$ with:

We assume the mean and variance do not change over time. The population autocorrleation that are s periods apart:

Sample Correlation

Population autocorrelations theoretically require a conceptual time series that goes on forever, from the infinite past and continuing to the infinite future. Sample autocorrelations are a sample of observations for a finite time period to estimate the population autocorrelations.

To estimate $\rho_{1}$:

More generally:

Because only $(T-s)$ observations are used to compute the numerator and $T$ observations are used to compute the denominator, an alternative calculation that leads to a larger estimate is:

Test of Significance

Assuming the null hypothesis as $H_{0}: \rho_{s} = 0$ is true, $r_{s}$ has an approximate normal distribution:

Standardizing $\hat{\rho}_{s}$:

Correlogram

Also called the sample ACF, the correlogram is the sequence of autocorrelations $r_{1}, r_{2}, \cdots$.

We say that $r_{s}$ is significantly different from 0 at a 5% significance if:

Example: Sample Autocorrelations for Unemployment

Consider the quarterly series for the U.S. unemployment rate found in the dataset usmacro.

The horizontal line is drawn at:

usmacro |>
  ACF(u) |>
  autoplot() +
  ggtitle("ACF") +  
  theme_tq()

Example: Sample Autocorrelations for GDP Growth Rate

The autocorrelations are smaller than those for the unemployment series, but there’s a series of pattern where the correlations oscillate between signifcance and insignificance. This could be due to the business cycle.

usmacro |>
  ACF(g, lag_max = 48) |>
  autoplot() +
  scale_x_continuous(
    breaks = seq(0, 48, 5)    
  )+
  ggtitle("ACF") +
  theme_tq()

Stationarity

Stationarity means the mean and variances do not change over time and autocorrelation between variables depend only on how far apart they are in time. Another way to think of this is if you take any subset of the time series, they will have the same mean, variance, and $\rho{1}, \rho_{2}, \cdots$. Up till now, we have assumed in the modeling that the variables are stationary.

Test of stationarity with “Unit Root Test” can be taken to detect non-stationarity which will cover later.

In addition to assuming that the variables are stationary, we also assume they are weakly dependent. Weak dependence implies that as $s \rightarrow \infty$, $x_{t}, x_{t+s}$ are becoming independent. In other words, as s grows larger, the autocorrelation decreases to zero. Stationary variables typically have weak dependence but not always.

Stationary variables need to be weakly dependent in order for the least squares estimator to be consistent.

Example: Are the Unemployment and Growth Rate Series Stationary and Weakly Dependent?

We will answer the question of stationarity in terms of formal test later when we cover unit root test. For now, we graph the ACFs:

usmacro |>
  ACF(u) |>
  autoplot() +
  ggtitle("ACF (Unemployment Rate)") +   
  theme_tq()

usmacro |>
  ACF(g) |>
  autoplot() +
  ggtitle("ACF (GDP Growth Rate)") +
  theme_tq()

We can see that the autocorrelations die out pretty quickly so we conclude it is weakly dependent. But we cannot ascertain whether it is stationary from the correlogram. It turns out that unemployment rate is indeed stationary.

GDP growth rate looks more stationary and certainly weakly dependent.

Knowing the unemployment and growth rates are stationary and weakly dependent means that we can proceed to use them for time-series regression models with stationary variables. With the exception of a special case known as cointegration, variables in time-series regressions must be stationary and weakly dependent for the least squares estimator to be consistent.

Forecasting

In this section, we consider forecasting using two different models, an AR model, and an ARDL model. The focus is on short-term forecasting, typically up to three periods into the future.

Suppose we have the following ARDL(2, 2) model:

Notice that the term $x_{t}$ is missing. The reason is it is unlikely that $x_{t}$ will be observed at the time of forecast so the omission is desirable.

Hence the ARDL forecast equation would be:

Defining the information set to be the set of all current and past observations of $y$ and $x$ at time $t$ as:

Assuming that we are standing at the end of the sample period, having observed $y_{T}, x_{T}$, the one-period ahead forecasting problem is to find a forecast $\hat{y}_{T+1}\mid I_{T}$.

The best forecast in a minimizing the conditional mean-squared forecast error :

Is the conditional expectation:

For example, suppose we believe only 2 lags of $y, x$ are relevant, we are assuming that:

By employing an ARDL(2, 2) model, we are assuming that, for forecasting $y_{T+1}$ , observations from more than two periods in the past do not convey any extra information relative to that contained in the most recent two observations.

Furthermore, we require:

For 2-period and 3-period ahead forecasts:

Note that we have estimate of $E[y_{T+2} \mid I_{T}], [y_{T+1} \mid I_{T}]$ from previous period’s forecasts, but $[x_{T+2} \mid I_{T}], [x_{T+1} \mid I_{T}]$ require extra information. They could be derived from independent forecasts or in what-if type scenarios. If the model is a pure AR model, this issue does not arise.

Example: Forecasting Unemployment with an AR(2) Model

To demonstrate how to use an AR model for forecasting, we consider this AR(2) model for forecasting US unemployment rate:

We assume that past values of unemployment cannot be used to forecast the error in the current period:

The expressions for forecasts for the remainder of 2016:

Example: OLS Estimation of the AR(2) Model for Unemployment

The assumption that $E[e_{t}\mid I_{t-1}] = 0$ is sufficient for the OLS estimator to be consistent. However, the OLS estimator will be biased, but consistency gives it a large-sample justification. The reason it is biased is because even though:

It does not mean future values will not be correlated with $e_{t}$, which is a condition for the estimator to be unbiased. In other words, the assumption of $E[e_{t}\mid I_{t-1}]$ is weaker than the strict exogeneity assumption.

The OLS estimates yields the following equation:

The standard error and $\hat{\sigma}$ estimates will be valid if the conditional homoskedasticity assumption is true:

We can show this in R:

fit <- usmacro |>
  model(
    ARIMA(
      u ~ 1 +
        lag(u, 1) +
        lag(u, 2) +
        pdq(0, 0, 0) +    
        PDQ(0, 0, 0)
    )
  )

> tidy(fit) |>
  select(-.model)
# A tibble: 3 × 5
  term      estimate std.error statistic   p.value
  <chr>        <dbl>     <dbl>     <dbl>     <dbl>
1 lag(u, 1)    1.61     0.0454     35.5  6.95e-104
2 lag(u, 2)   -0.662    0.0453    -14.6  4.80e- 36
3 intercept    0.289    0.0662      4.36 1.88e-  5

> report(fit)
Series: u 
Model: LM w/ ARIMA(0,0,0) errors 

Coefficients:
      lag(u, 1)  lag(u, 2)  intercept
         1.6128    -0.6621     0.2885
s.e.     0.0454     0.0453     0.0662

sigma^2 estimated as 0.08621:  log likelihood=-51.92     
AIC=111.84   AICc=111.99   BIC=126.27

Example: Unemployment Forecasts

Having now the OLS estimates, we can use it for forecasting:

Two quarters ahead forecast:

Two quarters ahead forecast:

Forecast Intervals and Standard Errors

Given the general ARDL(2, 2) model:

The one-period ahead forecast error $f_{1}$ is given by:

To simplify calculation, we ignore the error from estimating the coefficients. The coefficients error in most cases pales with comparison with the variance of the random error. Using this simplication we have:

For two-periods ahead forecast:

Assuming that the values for $\hat{x}_{T+1}$ are given in a what-if scenario:

And the actual two-period ahead:

Then the two-period ahead forecast error is:

For the estimated three-period ahead and assuming that the values for $\hat{x}_{T+1}$ are given in a what-if scenario:

And the actual three-period ahead:

Then the three-period ahead forecast error is:

For the estimated 4-period ahead:

And the actual 4-period ahead:

Then the 4-period ahead forecast error is:

Taking variance of the error we get:

Example: Forecast Intervals for Unemployment from the AR(2) Model

Given the following AR(2) model:

To forecast in R, we first fit the model using fable:

fit <- usmacro |>
  model(ARIMA(u ~ pdq(2, 0, 0) + PDQ(0, 0, 0)))    

> tidy(fit) |>
  select(-.model)
# A tibble: 3 × 5
  term     estimate std.error statistic   p.value
  <chr>       <dbl>     <dbl>     <dbl>     <dbl>
1 ar1         1.61     0.0449      35.9 2.09e-105
2 ar2        -0.661    0.0451     -14.7 2.44e- 36
3 constant    0.278    0.0174      16.0 4.77e- 41     

The coefficients are slightly different because fable uses MLE while the book uses OLS. As the forecast variance calculation is not directly supported:

fit <- usmacro |>
  model(
    ARIMA(
      u ~ 1 +
        lag(u, 1) +
        lag(u, 2) +
        pdq(0, 0, 0) +
        PDQ(0, 0, 0)
    )
  )

> tidy(fit) |>
  select(-.model)
# A tibble: 3 × 5
  term      estimate std.error statistic   p.value
  <chr>        <dbl>     <dbl>     <dbl>     <dbl>
1 lag(u, 1)    1.61     0.0454     35.5  6.95e-104
2 lag(u, 2)   -0.662    0.0453    -14.6  4.80e- 36
3 intercept    0.289    0.0662      4.36 1.88e-  5     

To forecast in R:

fcast <- fit |>
  forecast(h = 3) |>
  hilo(95) |>
  select(-.model)

> fcast
# A tsibble: 3 x 4 [1Q]
     date             u .mean                  `95%`
    <qtr>        <dist> <dbl>                 <hilo>
1 2016 Q2 N(4.9, 0.087)  4.87 [4.297622, 5.452135]95
2 2016 Q3  N(4.9, 0.31)  4.90 [3.804995, 5.995970]95
3 2016 Q4    N(5, 0.64)  4.96 [3.391705, 6.525075]95     

And the standard errors:

> sqrt(distributional::variance(fcast$u))      
[1] 0.2945241 0.5589325 0.7993436

And graphing the forecast:

fit |>
  forecast(h = 3) |>
  autoplot(usmacro) +   
  theme_tq()

Example: Forecasting Unemployment with an ARDL(2, 1) Model

We include a lagged value of the growth rate of the GDP. The OLS model is:

Unfortunately fable doesn’t support ARDL models directly. However we can do this using the dLagM package:

library(dLagM)

usmacro$g1 <- lag(usmacro$g)    

rem.p <- list(
  g1 = c(1)
)
rem.q <- c()

remove <- list(
  p = rem.p, q = rem.q
)

fit <- ardlDlm(
  u ~ g1,
  p = 1,
  q = 2,
  data = usmacro,
  remove = remove
)

> summary(fit)
Time series regression with "ts" data:
Start = 3, End = 273

Call:
dynlm(formula = as.formula(model.text), data = data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.83591 -0.17311 -0.02963  0.15226  1.20016 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.36157    0.07228   5.002 1.03e-06 ***
g1.t        -0.04824    0.01949  -2.475   0.0139 *  
u.1          1.53310    0.05555  27.597  < 2e-16 ***
u.2         -0.58179    0.05559 -10.465  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1     

Residual standard error: 0.2919 on 267 degrees of freedom
Multiple R-squared:  0.9685,	Adjusted R-squared:  0.9681 
F-statistic:  2734 on 3 and 267 DF,  p-value: < 2.2e-16

We can see that the above result matches the OLS estimates.

Using a what-if scenario, we assume that:

fcast <- dLagM::forecast(
  fit,
  x = c(0.31, 0.869, 1.069),
  h = 3,
  interval = TRUE
)

> fcast
$forecasts
    95% LB Forecast   95% UB
1 4.402419 4.949870 5.522786
2 4.139368 5.057540 6.107228
3 3.842399 5.183947 6.747698

$call
forecast.ardlDlm(model = fit,
    x = c(0.31, 0.869, 1.069),
    h = 3, interval = TRUE)

attr(,"class")
[1] "forecast.ardlDlm" "dLagM"   

To retrieve the standard errors:

> (fcast$forecasts[, 3] - fcast$forecasts[, 2]) / 1.96     
[1] 0.2923041 0.5355552 0.7978317

In fable, we can use the autocorrelation of the error to get similar coefficients. Note that the intercept is different from ARDL as the equation is fundamentally different. The coefficients are slightly different due to MLE vs OLS:

fit <- usmacro |>
  model(
    ARIMA(
      u ~ lag(g) +
        pdq(2, 0, 0) +
        PDQ(0, 0, 0)
    )
  )

> tidy(fit) |>
  select(-.model)
# A tibble: 4 × 5
  term      estimate std.error statistic   p.value
  <chr>        <dbl>     <dbl>     <dbl>     <dbl>
1 ar1        1.61       0.0454    35.5   4.25e-104
2 ar2       -0.659      0.0456   -14.5   1.52e- 35
3 lag(g)    -0.00574    0.0119    -0.482 6.30e-  1
4 intercept  5.76       0.362     15.9   1.04e- 40