Sparsity

Consider the linear model

and assume that , , are i.i.d. with a distribution and a density , where in a neighborhood of . Under some mild conditions

where and Refer to Koenker and Bassett (1982).

This asymptotic distribution for the regression quantile can be used to construct confidence intervals. However, the reciprocal of the density function

which is called the sparsity function, must first be estimated.

Since

can be estimated by the difference quotient of the empirical quantile function—that is,

where is an estimate of and is a bandwidth that tends to zero as .

The QUANTREG procedure provides two bandwidth methods. The Bofinger bandwidth

is an optimizer of mean squared error for standard density estimation, and the Hall-Sheather bandwidth

is based on Edgeworth expansions for studentized quantiles, where is the second derivative of and satisfies for the construction of confidence intervals. The quantity

is not sensitive to and can be estimated by assuming is Gaussian.

can be estimated by the empirical quantile function of the residuals from the quantile regression fit,

or the empirical quantile function of regression proposed by Bassett and Koenker (1982),

The QUANTREG procedure interpolates the first empirical quantile function and gets the piecewise linear version

is set to a constant if falls outside .

This estimator of the sparsity function is sensitive to the i.i.d. assumption. Alternately, Koenker and Machado (1999) considered the non-i.i.d. case. By assuming local linearity of the conditional quantile function in , they proposed a local estimator of the density function by using the difference quotient. A Huber sandwich estimate of the covariance and standard error is computed and used to construct the confidence intervals. One difficulty with this method is the selection of the bandwidth when using the difference quotient. With a small sample size, either the Bofinger or the Hall-Sheather bandwidth tends to be too large to assure local linearity of the conditional quantile function. The QUANTREG procedure uses a heuristic bandwidth selection in these cases.

By default, the QUANTREG procedure computes non-i.i.d. confidence intervals. You can request i.i.d. confidence intervals with the IID option in the PROC statement.

Inversion of Rank Tests

The classical theory of rank tests can be extended to test the hypothesis : in the linear regression model . Here . See Gutenbrunner and Jureckova (1992) for more details. By inverting this test, confidence intervals can be computed for the regression quantiles corresponding to .

The rank score function can be obtained by solving the dual problem

For a fixed quantile , integrating with respect to the -quantile score function

yields the -quantile scores

Under the null hypothesis :

for large , where .

Let

then from the constraint in the full model. In order to obtain confidence intervals for , a critical value can be specified for . The dual vector is a piecewise constant in , and can be altered without compromising the optimality of as long as the sign of the residuals in the primal quantile regression problem do not change. When gets to such a boundary the solution does change, but can be restored by taking one simplex pivot. The process can continue in this way until exceeds the specified critical value. Since is piecewise constant, interpolation can be used to obtain the desired level of confidence interval; see Koenker and d’Orey (1994).

Resampling

The bootstrap can be implemented to compute confidence intervals for regression quantile estimates. As in other regression applications, both the residual bootstrap and the -pair bootstrap can be used. The former assumes i.i.d. random errors and resamples from the residuals, while the later resamples pairs and accommodates some forms of heteroscedasticity. Koenker (1994) considered a more interesting resampling mechanism, resampling directly from the full regression quantile process, which he called the Heqf bootstrap.

In contrast with these bootstrap methods, Parzen, Wei, and Ying (1994) observed that

which is the estimating equation for the th regression quantile, is a pivotal quantity for the th quantile regression parameter . In other words, the distribution of can be generated exactly by a random vector , which is a weighted sum of independent, re-centered Bernoulli variables. They further showed that for large , the distribution of can be approximated by the conditional distribution of , where solves an augmented quantile regression problem with observations with and sufficiently large for a given realization of . By exploiting the asymptotically pivot role of the quantile regression "gradient condition," this approach also achieves some robustness to certain heteroscedasticity.

Although the bootstrap method by Parzen, Wei, and Ying (1994) is much simpler, it is too time-consuming for relatively large data sets, especially for high-dimensional data sets. The QUANTREG procedure implements a new, general resampling method developed by He and Hu (2002), which is referred to as the Markov chain marginal bootstrap (MCMB). For quantile regression, the MCMB method has the advantage that it solves one-dimensional equations instead of -dimensional equations, as do the previous bootstrap methods. This greatly improves the feasibility of the resampling method in computing confidence intervals for regression quantiles. Since resampling methods achieve stability only for relatively large data sets, they are not recommended for small data sets ( and ).