Example 4.2: Switching Regression Example

The MODEL Procedure

Example 4.2: Switching Regression Example

Take the usual linear regression problem

$y \;=\; X \beta \; + \; u$

where Y denotes the n column vector of the dependent variable, X denotes the (n × k) matrix of independent variables, $\beta$ denotes the k column vector of coefficients to be estimated, n denotes the number of observations (i=1,2,...,n), and k denotes the number of independent variables.

You can take this basic equation and split it into two regimes, where the ith observation on y is generated by one regime or the other.

$y_{i} \;=\; \sum_{j=1}^k \, \beta_{1j} X_{ji} \; + \; u_{1i} \;=\; x_{i}' \b... ...}^k \, \beta_{2j} X_{ji} \; + \; u_{2i} \;=\; x_{i}' \beta_{2} \; + \; u_{2i}$

where x_hi and x_hj are the ith and jth observations, respectively, on x_h. The errors, u_1i and u_2i, are assumed to be distributed normally and independently, with mean zero and constant variance. The variance for the first regime is $\sigma_{1}^2$ , and the variance for the second regime is $\sigma_{2}^2$ . If $\sigma_{1}^2 \neq \sigma_{2}^2$ and $\beta_{1} \neq \beta_{2}$ ,the regression system given previously is thought to be switching between the two regimes.

The problem is to estimate $\beta_{1}$ , $\beta_{2}$ , $\sigma_{1}$ , and $\sigma_{2}$ without knowing a priori which of the n values of the dependent variable, y, was generated by which regime. If it is known a priori which observations belong to which regime, a simple Chow test can be used to test $\sigma_{1}^2=\sigma_{2}^2$ and $\beta_{1}=\beta_{2}$ .

Using Goldfeld and Quandt's D-method for switching regression, you can solve this problem. Assume that there exists observations on some exogenous variables z_1i, z_2i, ..., z_pi, where z determines whether the ith observation is generated from one equation or the other.

$y_{i} &=& x_{i}' \beta_{1} + u_{1i} {if} \sum_{j=1}^p \, \pi_{j} z_{ji} \; \le... ...&=& x_{i}' \beta_{2} + u_{2i} {if} \sum_{j=1}^p \, \pi_{j} z_{ji} \; \gt \; 0$

where $\pi_{j}$ are unknown coefficients to be estimated. Define d(z_i) as a continuous approximation to a step function. Replacing the unit step function with a continuous approximation using the cumulative normal integral enables a more practical method that produces consistent estimates.

$d(z_{i}) \;=\; \frac{1}{\sqrt{2 \pi} \sigma} \, \int_{- \infty}^{\sum \pi_{j} z_{ji}} \, exp [ - \frac{1}2 \, \frac{\xi^2}{\sigma^2} ] \, d \xi$

D is the n dimensional diagonal matrix consisting of d(z_i).

$D \;=\; [ d(z_{1}) & 0 & 0 & 0 \0 & d(z_{2}) & 0 & 0 \0 & 0 & \ddots & 0 \0 & 0 & 0 & d(z_{n})]$

The parameters to estimate are now the k $\beta_{1}$ 's, the k $\beta_{2}$ 's, $\sigma_{1}^2$ , $\sigma_{2}^2$ , p $\pi$ 's, and the ${{\sigma}}$ introduced in the d(z_i) equation. The ${{\sigma}}$ can be considered as given a priori, or it can be estimated, in which the estimated magnitude provides an estimate of the success in discriminating between the two regimes (Goldfeld and Quandt 1976).

$Y=(I \, - \, D) \, X \beta_{1} + D X \beta_{2} + W$

where W = (I - D) U₁ + D U₂, and W is a vector of unobservable and heteroscedastic error terms. The covariance matrix of W is denoted by $\Omega$ , where $\Omega \,=\, (I-D)^2 \sigma_{1}^2 \, + \, D^2 \sigma_{2}^2$ . The maximum likelihood parameter estimates maximize the following log-likelihood function.

$logL &=& - \frac{n}2 \log 2 \pi - \frac{1}2 \log | \Omega | - \& & \frac{1}2 ... ...} - D X \beta_{2} ] ' \Omega^{-1} [ Y - (I - D) X \beta_{1} - D X \beta_{2} ] ]$

As an example, you now can use this switching regression likelihood to develop a model of housing starts as a function of changes in mortgage interest rates. The data for this example is from the U.S. Census Bureau and covers the period from January 1973 to March 1999. The hypothesis is that there will be different coefficients on your model based on whether the interest rates are going up or down.

So the model for z_i will be the following

z_i = p * ( rate_i - rate_i-1 )

where rate_i is the mortgage interest rate at time i and p is a scale parameter to be estimated.

The regression model will be the following

$starts_i &=& intercept_1 + ar1 * starts_{i-1} + djf1 * decjanfeb z_i \lt 0 \starts_i &=& intercept_2 + ar2 * starts_{i-1} + djf2 * decjanfeb z_i \gt=0$

where starts_i is the number of housing starts at month i and decjanfeb is a dummy variable indicating that the current month is one of December, January, or February.

This model is written using the following SAS statements.

proc model data=switch;
   parms sig1=10 sig2=10 int1 b11 b13 int2 b21 b23 p;
   bounds 0.0001 < sig1 sig2;

   a = p*dif(rate);       /* Upper bound of integral */
   d = probnorm(a);       /* Normal CDF as an approx of switch */

                          /* Regime 1 */
   y1 = int1 + zlag(starts)*b11 + decjanfeb *b13 ;     
                          /* Regime 2 */
   y2 = int2 + zlag(starts)*b21 + decjanfeb *b23 ;     
                          /* Composite regression equation */
   starts  = (1 - d)*y1 +  d*y2;     

                         /* Resulting log-likelihood function */
   logL = (1/2)*( (318*log(2*3.1415)) +
        log( (sig1**2)*((1-d)**2)+(sig2**2)*(d**2) )
       + (resid.starts*( 1/( (sig1**2)*((1-d)**2)+
        (sig2**2)*(d**2) ) )*resid.starts) ) ;

   errormodel starts ~ general(logL);

fit starts / method=marquardt converge=1.0e-5;

     /* Test for significant differences in the parms */
test int1 = int2 ,/ lm;
test b11 = b21 ,/ lm;
test b13 = b23 ,/ lm;
test sig1 = sig2 ,/ lm;

run;

Four TEST statements were added to test the hypothesis that the parameters were the same in both regimes. The parameter estimates and ANOVA table from this run are shown in Output 4.2.1.

Output 4.2.1: Parameter Estimates from the Switching Regression

The MODEL Procedure

Nonlinear Liklhood Summary of Residual Errors
Equation	DF Model	DF Error	SSE	MSE	Root MSE	R-Square	Adj R-Sq
starts	9	304	85877.9	282.5	16.8075	0.7806	0.7748

Nonlinear Liklhood Parameter Estimates
Parameter	Estimate	Approx Std Err	t Value	Approx Pr > \|t\|
sig1	15.47451	0.9475	16.33	<.0001
sig2	19.77797	1.2710	15.56	<.0001
int1	32.82232	5.9070	5.56	<.0001
b11	0.739529	0.0444	16.65	<.0001
b13	-15.456	3.1909	-4.84	<.0001
int2	42.73243	6.8153	6.27	<.0001
b21	0.734112	0.0477	15.37	<.0001
b23	-22.5178	4.2979	-5.24	<.0001
p	25.94332	8.5181	3.05	0.0025

The test results shown in Output 4.2.2 suggest that the variance of the housing starts, SIG1 and SIG2, are significantly different in the two regimes. The tests also show a significant difference in the AR term on the housing starts.

Output 4.2.2: Parameter Estimates from the Switching Regression

The MODEL Procedure

Test Results
Test	Type	Statistic	Pr > ChiSq	Label
Test0	L.M.	0.02	0.8810	int1 = int2
Test1	L.M.	240001	<.0001	b11 = b21
Test2	L.M.	0.02	0.8933	b13 = b23
Test3	L.M.	319354	<.0001	sig1 = sig2

Chapter Contents
Previous
Next
Top