The following SAS^® procedures implement these methods in the context of the analyses that they perform:

In SAS/STAT^® software:

PROC MULTTEST can use bootstrap or permutation resampling (see the BOOTSTRAP and PERMUTATION options in the PROC MULTTEST statement) to adjust the p-values for the tests that are computed by the procedure.

PROC DISCRIM uses cross validation (see the CROSSVALIDATE, CROSSLIST, and CROSSLISTERR options in the PROC DISCRIM statement) to obtain nearly unbiased estimates of the classification error rates in a discriminant analysis.

PROC LOGISTIC uses a one-step approximation of cross validation to obtain predicted probabilities. See the PREDPROBS=CROSSVALIDATE option in the OUTPUT statement and the CTABLE option in the MODEL statement.

PROC SURVEYSELECT can create multiple bootstrap (with replacement) or permutation (without replacement) resamples using the REP= option to create independent, replicated samples from a data set. Use the METHOD= option to select sampling with or without replacement and the SAMPSIZE= option to control the size of the samples.

The GAM, LOESS and TPSPLINE procedures can use cross validation to choose the smoothing parameter. See the METHOD=GCV option in the MODEL statement of PROC GAM and the SELECT= option in PROC LOESS. PROC TPSPLINE uses cross validation by default.

PROC PLS enables you to choose the number of extracted factors by cross validation. See the CV= option in the PROC PLS statement.

PROC MODECLUS can use likelihood cross validation to choose the smoothing parameter. See the CROSS and CROSSLIST options in the PROC MODECLUS statement.

PROC CALIS computes the expected cross validation index (ECVI) that measures how good a model is for predicting future sample covariances.

PROC MI can use bootstrap resampling, which uses a simple random sample with replacement from the input data set for the initial estimate or to obtain overdispersed starting values for multiple chains. See the INITIAL=EM(BOOTSTRAP) option in the MCMC statement. Also, the propensity score method applies an approximate Bayesian bootstrap imputation. See the PROPENSITY option in the MONOTONE statement.

PROC GLMSELECT provides leave-one-out and k-fold cross validation for estimating prediction error. Cross validation can be used as the selection criterion for selecting model effects, as a stopping rule for the selection process, and as the criterion for final model determination.

PROC QUANTREG implements the Markov chain marginal bootstrap (MCMB) general resampling method of He and Hu (2002) to provide confidence intervals for regression quantile estimates. These intervals, available with the CI=RESAMPLING option in the PROC QUANTREG statement, provide some robustness to heteroscedasticity.

The SURVEYMEANS, SURVEYREG, SURVEYLOGISTIC, SURVEYPHREG, and SURVEYFREQ procedures offer jackknife variance estimation.

PROC ADAPTIVEREG, beginning in SAS^® 9.3M2 (TS1M2), provides leave-one-out and k-fold cross validation.

PROC HPSPLIT, beginning in SAS^® 9.4M3 (TS1M3), can (in the absence of validation data) apply k-fold cross validation to cost-complexity pruning to select a subtree that generalizes well and does not overfit the training data. It can also do model assessment with cross validation and can provide a confusion matrix.

PROC NLIN, beginning in SAS^® 9.4M1 (TS1M1), with the BOOTSTRAP statement requests bootstrap estimation of confidence intervals for parameters and bootstrap estimates of the covariance matrix and the correlation matrix of the parameter estimates.

PROC TTEST, beginning in SAS^® 9.4M5 (TS1M5), with the BOOTSTRAP statement provides bootstrap standard error, bias estimates, and confidence limits for means and standard deviations in one-sample, paired, and two-sample designs.

The ​​​​​CAUSALMED and CAUSALTRT procedures, beginning in SAS 9.4M5, with the BOOTSTRAP statement provides bootstrap estimates of standard errors and bootstrap confidence intervals for various effects and percentages of total effects.

The SURVEYMEANS, SURVEYREG, SURVEYLOGISTIC, SURVEYPHREG, SURVEYIMPUTE and SURVEYFREQ procedures, beginning in SAS 9.4M5, with the VARMETHOD=BOOTSTRAP option provide variance estimation by the bootstrap method.

Also, there are macros (SAS Note 24982) available that perform bootstrap and jackknife analysis for simple random samples, computing approximate standard errors, bias-corrected estimates, and confidence intervals assuming a normal sampling distribution. In order to use these macros, you need to know enough about the SAS macro language to be able to write simple macros. For more information about bootstrapping and capabilities in SAS, see this blog entry.

Operating System and Release Information