Time Series Analysis and Examples |
Consider the time series
:
![y_t = f(t) + \epsilon_t](images/timeseriesexpls_timeseriesexplseq102.gif)
where
![f(t)](images/timeseriesexpls_timeseriesexplseq103.gif)
is an unknown smooth function and
![\epsilon_t](images/timeseriesexpls_timeseriesexplseq104.gif)
is an
![iid](images/timeseriesexpls_timeseriesexplseq105.gif)
random variable with
zero mean and positive variance
![\sigma^2](images/timeseriesexpls_timeseriesexplseq106.gif)
.
Whittaker (1923) provides the solution, which
balances a tradeoff between closeness to the data
and the
![k](images/timeseriesexpls_timeseriesexplseq107.gif)
th-order difference equation.
For a fixed value of
![\lambda](images/timeseriesexpls_timeseriesexplseq108.gif)
and
![k](images/timeseriesexpls_timeseriesexplseq107.gif)
, the solution
![\hat{f}](images/timeseriesexpls_timeseriesexplseq109.gif)
satisfies
![\min_f \sum_{t=1}^t \{ [y_t - f(t)]^2 + \lambda^2 [\nabla^k f(t)]^2 \}](images/timeseriesexpls_timeseriesexplseq110.gif)
where
![\nabla^k](images/timeseriesexpls_timeseriesexplseq111.gif)
denotes the
![k](images/timeseriesexpls_timeseriesexplseq107.gif)
th-order difference operator.
The value of
![\lambda](images/timeseriesexpls_timeseriesexplseq108.gif)
can be viewed
as the smoothness tradeoff measure.
Akaike (1980a) proposed the Bayesian
posterior PDF to solve this problem.
![\ell(f) = \exp \{ -\frac{1}{2 \sigma^2} \sum_{t=1}^t [y_t - f(t)]^2 \} \exp \{ -\frac{\lambda^2}{2 \sigma^2} \sum_{t=1}^t [\nabla^k f(t)]^2 \}](images/timeseriesexpls_timeseriesexplseq112.gif)
Therefore, the solution can be obtained
when the function
![\ell(f)](images/timeseriesexpls_timeseriesexplseq113.gif)
is maximized.
Assume that time series is decomposed as follows:
![y_t = t_t + s_t + \epsilon_t](images/timeseriesexpls_timeseriesexplseq33.gif)
where
![t_t](images/timeseriesexpls_timeseriesexplseq19.gif)
denotes the trend component
and
![s_t](images/timeseriesexpls_timeseriesexplseq20.gif)
is the seasonal component.
The trend component follows the
![k](images/timeseriesexpls_timeseriesexplseq107.gif)
th-order
stochastically perturbed difference equation.
![\nabla^k t_t = w_{1t}, \hspace*{0.25in} w_{1t} \sim n(0,\tau_1^2)](images/timeseriesexpls_timeseriesexplseq114.gif)
For example, the polynomial trend
component for
![k=2](images/timeseriesexpls_timeseriesexplseq115.gif)
is written as
![t_t = 2t_{t-1} - t_{t-2} + w_{1t}](images/timeseriesexpls_timeseriesexplseq116.gif)
To accommodate regular seasonal effects, the
stochastic seasonal relationship is used.
![\sum_{i=0}^{l-1} s_{t-i} = w_{2t} \hspace*{0.25in} w_{2t} \sim n(0,\tau_2^2)](images/timeseriesexpls_timeseriesexplseq117.gif)
where
![l](images/timeseriesexpls_timeseriesexplseq118.gif)
is the number of seasons within a period.
In the context of Whittaker and Akaike, the smoothness
priors problem can be solved by the maximization of
![\ell(f) & = & \exp [ -\frac{1}{2 \sigma^2} \sum_{t=1}^t (y_t - t_t - s_t)^2 ... ...\frac{\tau_2^2}{2 \sigma^2} \sum_{t=1}^t ( \sum_{i=0}^{l-1} s_{t-i} )^2 ]](images/timeseriesexpls_timeseriesexplseq119.gif)
The values of hyperparameters
![\tau_1^2](images/timeseriesexpls_timeseriesexplseq120.gif)
and
![\tau_2^2](images/timeseriesexpls_timeseriesexplseq121.gif)
refer to a measure of uncertainty of prior information.
For example, the large value of
![\tau_1^2](images/timeseriesexpls_timeseriesexplseq120.gif)
implies a relatively smooth trend component.
The ratio
![\frac{\tau_i^2}{\sigma^2}\; (i=1,2)](images/timeseriesexpls_timeseriesexplseq122.gif)
can be considered as a signal-to-noise ratio.
Kitagawa and Gersch (1984) use the Kalman filter recursive
computation for the likelihood of the tradeoff parameters.
The hyperparameters are estimated by combining
the grid search and optimization method.
The state space model and Kalman filter recursive
computation are discussed in the section "State Space and Kalman Filter Method".