Time Series Analysis and Examples: Nonstationary Time Series

Time Series Analysis and Examples

Nonstationary Time Series

The subroutines TSMLOCAR, TSMLOMAR, and TSTVCAR are used to analyze nonstationary time series models. The AIC statistic is extensively used to analyze the locally stationary model.

Locally Stationary AR Model

When the time series is nonstationary, the TSMLOCAR (univariate) and TSMLOMAR (multivariate) subroutines can be employed. The whole span of the series is divided into locally stationary blocks of data, and then the TSMLOCAR and TSMLOMAR subroutines estimate a stationary AR model by using the least squares method on this stationary block. The homogeneity of two different blocks of data is tested by using the AIC.

Given a set of data $\{y_1, ... ,y_t\}$ , the data can be divided into blocks of sizes , where , and and are unknown. The locally stationary model is fitted to the data

$y_t = \alpha_0^i + \sum_{j=1}^{p_i} \alpha_j^i y_{t-j} + \epsilon_t^i$

where

$t_{i-1} = \sum_{j=1}^{i-1} t_j \lt t \leq t_i = \sum_{j=1}^i t_j \hspace*{0.25in}{ for } i=1, ... ,k$

where $\epsilon_t^i$ is a Gaussian white noise with ${\rm e} \epsilon_t^i = 0$ and ${\rm e}(\epsilon_t^i)^2 = \sigma_i^2$ . Therefore, the log-likelihood function of the locally stationary series is

$\ell = -\frac{1}2\sum_{i=1}^k [ t_i\log(2 \pi \sigma_i^2) + \frac{1}{\sigma_i^... ...1}^{t_i} ( y_t - \alpha_0^i - \sum_{j=1}^{p_i} \alpha_j^i y_{t-j} )^2 ]$

Given $\alpha_j^i$ ,

, the maximum of the log-likelihood function is attained at

$\hat{\sigma}_i^2 = \frac{1}{t_i} \sum_{t=t_{i-1}+1}^{t_i} ( y_t - \hat{\alpha}_0^i - \sum_{j=1}^{p_i} \hat{\alpha}_j^i y_{t-j} )^2$

The concentrated log-likelihood function is given by

$\ell^* = - \frac{t}2[1 + \log(2 \pi)] - \frac{1}2 \sum_{i=1}^k t_i \log(\hat{\sigma}_i^2)$

Therefore, the maximum likelihood estimates, $\hat{\alpha}_j^i$ and $\hat{\sigma}_i^2$ , are obtained by minimizing the following local SSE:

${\rm sse} = \sum_{t=t_{i-1}+1}^{t_i} ( y_t - \hat{\alpha}_0^i - \sum_{j=1}^{p_i} \hat{\alpha}_j^i y_{t-j} )^2$

The least squares estimation of the stationary model is explained in the section "Least Squares and Householder Transformation".

The AIC for the locally stationary model over the pooled data is written as

$\sum_{i=1}^k t_i \log(\hat{\sigma}_i^2) + 2\sum_{i=1}^k (p_i + {intercept}+1)$

where intercept = 1 if the intercept term $(\alpha_0^i)$ is estimated; otherwise, intercept = 0. The number of stationary blocks (

), the size of each block (

), and the order of the locally stationary model is determined by the AIC. Consider the autoregressive model fitted over the block of data, $\{y_1, ... ,y_t\}$ , and let this model

be an AR(

) process. When additional data, $\{y_{t+1}, ... ,y_{t+t_1}\}$ , are available, a new model

, an AR(

) process, is fitted over this new data set, assuming that these data are independent of the previous data. Then AICs for models

and

are defined as

${\rm aic}_1 & = & t \log(\sigma_1^2) + 2(p_1 + {intercept} + 1) \ {\rm aic}_2 & = & t_1 \log(\sigma_2^2) + 2(p_2 + {intercept} + 1)$

The joint model AIC for

and

is obtained by summation

${\rm aic}_j = {\rm aic}_1 + {\rm aic}_2$

When the two data sets are pooled and estimated over the pooled data set, $\{y_1, ... ,y_{t+t_1}\}$ , the AIC of the pooled model is

${\rm aic}_a = (t + t_1) \log(\hat{\sigma}_a^2) + 2(p_a + {intercept} + 1)$

where $\sigma^2_a$ is the pooled error variance and

is the order chosen to fit the pooled data set.

Decision

If ${\rm aic}_j \lt {\rm aic}_a$ , switch to the new model, since there is a change in the structure of the time series.
If ${\rm aic}_j\geq {\rm aic}_a$ , pool the two data sets, since two data sets are considered to be homogeneous.

If new observations are available, repeat the preceding steps to determine the homogeneity of the data. The basic idea of locally stationary AR modeling is that, if the structure of the time series is not changed, you should use the additional information to improve the model fitting, but you need to follow the new structure of the time series if there is any change.

Time-Varying AR Coefficient Model

Another approach to nonstationary time series, especially those that are nonstationary in the covariance, is time-varying AR coefficient modeling. When the time series is nonstationary in the covariance, the problem in modeling this series is related to an efficient parameterization. It is possible for a Bayesian approach to estimate the model with a large number of implicit parameters of the complex structure by using a relatively small number of hyperparameters.

The TSTVCAR subroutine uses smoothness priors by imposing stochastically perturbed difference equation constraints on each AR coefficient and frequency response function. The variance of each AR coefficient distribution constitutes a hyperparameter included in the state space model. The likelihood of these hyperparameters is computed by the Kalman filter recursive algorithm.

The time-varying AR coefficient model is written

$y_t = \sum_{i=1}^m \alpha_{it} y_{t-i} + \epsilon_t$

where time-varying coefficients $\alpha_{it}$ are assumed to change gradually with time. The following simple stochastic difference equation constraint is imposed on each coefficient:

$\nabla^k \alpha_{it} = w_{it}, \hspace*{0.25in} w_{it} \sim n(0,\tau^2), \hspace*{0.25in} i=1, ... ,m$

The frequency response function of the AR process is written

$a(f) = 1 - \sum_{j=1}^m \alpha_{jt} \exp(-2 \pi jif)$

The smoothness of this function can be measured by the

th derivative smoothness constraint,

$r_k = \int_{-1/2}^{1/2} | \frac{d^k a(f)}{df^k} |^2 df = (2 \pi)^{2k} \sum_{j=1}^m j^{2k} \alpha_{jt}^2$

Then the TSTVCAR call imposes zero and second derivative smoothness constraints. The time-varying AR coefficients are the solution of the following constrained least squares:

$\sum_{t=1}^t ( y_t - \sum_{i=1}^m \alpha_{it} y_{t-i} )^2 + \tau^2 \sum_{t=1}^... ...\sum_{i=1}^m i^2 \alpha_{it}^2 + \nu^2 \sum_{t=1}^t \sum_{i=1}^m \alpha_{it}^2$

where $\tau^2$ , $\lambda^2$ , and $\nu^2$ are hyperparameters of the prior distribution.

Using a state space representation, the model is

${x}_t & = & {f}{x}_{t-1} + {g}{w}_t \ y_t & = & {h}_t{x}_t + \epsilon_t$

where

${x}_t & = & (\alpha_{1t}, ... ,\alpha_{mt}, ... , \alpha_{1,t-k+1}, ... ,\alph... ...{w}_t \ \epsilon_t ] & \sim & n (0, [ \tau^2{i}& 0 \ 0 & \sigma^2 ] )$

The computation of the likelihood function is straightforward. See the section "State Space and Kalman Filter Method" for the computation method.

Top of Page