The QUANTREG Procedure

Quantile Regression as an Optimization Problem

The model for linear quantile regression is

\[ \mb{y} = \bA ^{\prime }\bbeta + \bepsilon \]

where $\mb{y}=(y_1,\ldots ,y_ n)^{\prime }$ is the $(n \times 1)$ vector of responses, $\bA ^{\prime }=(\mb{x}_1,\ldots ,\mb{x}_ n)^{\prime }$ is the $(n \times p)$ regressor matrix, $\bbeta = (\beta _1,\ldots ,\beta _ p)^{\prime }$ is the $(p \times 1)$ vector of unknown parameters, and $\bepsilon = (\epsilon _1,\ldots ,\epsilon _ n)^{\prime }$ is the $(n \times 1)$ vector of unknown errors.

$L_1$ regression, also known as median regression, is a natural extension of the sample median when the response is conditioned on the covariates. In $L_1$ regression, the least absolute residuals estimate ${\hat\bbeta }_{\mathit{LAR}}$, referred to as the $L_1$-norm estimate, is obtained as the solution of the following minimization problem:

\[ \min _{\bbeta \in \mb{R}^ p} \sum _{i=1}^ n | y_ i - \mb{x}_ i^{\prime }\bbeta | \]

More generally, for quantile regression Koenker and Bassett (1978) defined the $\tau $ regression quantile, $0<\tau <1$, as any solution to the following minimization problem:

\[ \min _{\bbeta \in \mb{R}^ p} \left[\sum _{i\in \{ i: y_ i\geq \mb{x}_ i^{\prime }\bbeta \} } \tau |y_ i-\mb{x}_ i^{\prime }\bbeta | + \sum _{i\in \{ i: y_ i< \mb{x}_ i^{\prime }\bbeta \} } (1-\tau ) |y_ i-\mb{x}_ i^{\prime }\bbeta |\right] \]

The solution is denoted as $\hat{\bbeta }(\tau )$, and the $L_1$-norm estimate corresponds to $\hat{\bbeta }(1\slash 2)$. The $\tau $ regression quantile is an extension of the $\tau $ sample quantile $\hat\xi (\tau )$, which can be formulated as the solution of

\[ \min _{\xi \in \mb{R}} \left[ \sum _{i\in \{ i: y_ i\geq \xi \} } \tau |y_ i-\xi | + \sum _{i\in \{ i: y_ i< \xi \} } (1-\tau ) |y_ i-\xi | \right] \]

If you specify weights $w_ i, i=1,\ldots ,n$, with the WEIGHT statement, weighted quantile regression is carried out by solving

\[ \min _{\bbeta _ w \in \mb{R}^ p} \left[\sum _{i\in \{ i: y_ i\geq \mb{x}_ i^{\prime }\bbeta _ w\} } w_ i \tau |y_ i-\mb{x}_ i^{\prime }\bbeta _ w| + \sum _{i\in \{ i: y_ i< \mb{x}_ i^{\prime }\bbeta _ w\} } w_ i (1-\tau ) |y_ i-\mb{x}_ i^{\prime }\bbeta _ w|\right] \]

Weighted regression quantiles $\bbeta _ w$ can be used for L-estimation (Koenker and Zhao, 1994).