The experimental ENTROPY procedure implements a parametric method of linear estimation based on generalized maximum entropy.
Often the statistical-economic model of interest is ill-posed or under-determined for the observed data (for example, when limited data is available or acquiring data is costly). For the general linear model, this can imply that high degrees of collinearity exist among explanatory variables or that there are more parameters to estimate than observations to estimate them with. These conditions lead to high variances or non-estimability for traditional GLS estimates.
The principle of maximum entropy, at the base of the ENTROPY procedure, is the foundation for an estimation methodology that is characterized by its robustness to ill-conditioned designs and its ability to fit over-parameterized models.
Generalized maximum entropy, GME, is a means of selecting among probability distributions so as to choose the distribution that maximizes uncertainty or uniformity that remains in the distribution, subject to information that is already known about the distribution itself. Information takes the form of data or moment constraints in the estimation procedure. PROC ENTROPY creates a GME distribution for each parameter in the linear model, based upon support points that you supply. The mean of each distribution is used as the estimate of the parameter. Estimates tend to be biased, as they are a type of shrinkage estimate, but typically portray smaller variances than OLS counterparts, which makes them more desirable from a mean squared error viewpoint.
PROC ENTROPY can be used to fit simultaneous systems of linear regression models, Markov models, and seemingly unrelated regression models, as well as to solve pure inverse problems and unordered, multinomial choice problems. Bounds and restrictions on parameters can be specified and Wald, likelihood ratio, and Lagrange multiplier tests can be computed. Prior information can also be supplied to enhance estimates and data.
