The HPSEVERITY Procedure

An Example with Left-Truncation and Right-Censoring

PROC HPSEVERITY enables you to specify that the response variable values are left-truncated or right-censored. The following DATA step expands the data set of the previous example to simulate a scenario that is typically encountered by an automobile insurance company. The values of the variable Y represent the loss values on claims that are reported to an auto insurance company. The variable THRESHOLD records the deductible on the insurance policy. If the actual value of Y is less than or equal to the deductible, then it is unobservable and does not get recorded. In other words, THRESHOLD specifies the left-truncation of Y. LIMIT records the policy limit. If the value of Y is equal to or greater than the recorded value, then the observation is right-censored.

/*----- Lognormal Model with left-truncation and censoring -----*/
data test_sev2(keep=y threshold limit
        label='A Lognormal Sample With Censoring and Truncation');
   set test_sev1;
   label y='Censored & Truncated Response';
   if _n_ = 1 then call streaminit(45679);

   /* make about 20% of the observations left-truncated */
   if (rand('UNIFORM') < 0.2) then
      threshold = y * (1 - rand('UNIFORM'));
   else
      threshold = .;
   /* make about 15% of the observations right-censored */
   iscens = (rand('UNIFORM') < 0.15);
   if (iscens) then
      limit = y;
   else
      limit = .;
run;

The following statements use the AICC criterion to analyze which of the four predefined distributions (lognormal, Burr, gamma, and Weibull) has the best fit for the data:

proc hpseverity data=test_sev2 crit=aicc print=all ;
   loss y / lt=threshold rc=limit;

   dist logn burr gamma weibull;
   performance nthreads=2;
run;

The LOSS statement specifies the left-truncation and right-censoring variables. The DIST statement specifies the candidate distributions. The PRINT= option in the PROC HPSEVERITY statement requests that all the displayed output be prepared. The NTHREADS option in the PERFORMANCE statement specifies that two threads of computation be used. The option is shown here just for illustration. You should use it only when you want to restrict the procedure to use a different number of threads than the value of the CPUCOUNT= system option, which usually defaults to the number of physical CPU cores available on your machine, thereby allowing the procedure to fully utilize the computational power of your machine.

Some of the key results prepared by PROC HPSEVERITY are shown in Figure 9.4 through Figure 9.7. In addition to the estimates of the range, mean, and standard deviation of Y, the Descriptive Statistics for Variable y table shown in Figure 9.4 also indicates the number of observations that are left-truncated or right-censored. The Model Selection table in Figure 9.4 shows that models with all the candidate distributions have converged and that the Logn (lognormal) model has the best fit for the data according to the AICC criterion.

Figure 9.4: Summary Results for the Truncated and Censored Data

The HPSEVERITY Procedure

Input Data Set
Name WORK.TEST_SEV2
Label A Lognormal Sample With Censoring and Truncation

Descriptive Statistics for y
Observations 100
Observations Used for Estimation 100
Minimum 2.30264
Maximum 8.34116
Mean 4.62007
Standard Deviation 1.23627
Left Truncated Observations 23
Right Censored Observations 14

Model Selection
Distribution Converged AICC Selected
Logn Yes 298.92672 Yes
Burr Yes 302.66229 No
Gamma Yes 299.45293 No
Weibull Yes 309.26779 No


PROC HPSEVERITY also prepares a table that shows all the fit statistics for all the candidate models. It is useful to see which model would be the best fit according to each of the criteria. The All Fit Statistics table prepared for this example is shown in Figure 9.5. It indicates that the lognormal model is chosen by all the criteria.

Figure 9.5: Comparing All Statistics of Fit for the Truncated and Censored Data

All Fit Statistics
Distribution -2 Log
Likelihood
AIC AICC BIC KS AD CvM
Logn 294.80301 * 298.80301 * 298.92672 * 304.01335 * 0.51824 * 0.34736 * 0.05159 *
Burr 296.41229   302.41229   302.66229   310.22780   0.66984   0.36712   0.05726  
Gamma 295.32921   299.32921   299.45293   304.53955   0.62511   0.42921   0.05526  
Weibull 305.14408   309.14408   309.26779   314.35442   0.93307   1.40699   0.17465  
Note: The asterisk (*) marks the best model according to each column's criterion.


Specifying Initial Values for Parameters

All the predefined distributions have parameter initialization functions built into them. For the current example, Figure 9.6 shows the initial values that are obtained by the predefined method for the Burr distribution. It also shows the summary of the optimization process and the final parameter estimates.

Figure 9.6: Burr Model Summary for the Truncated and Censored Data

Initial Parameter Values and Bounds
Parameter Initial
Value
Lower
Bound
Upper
Bound
Theta 4.78102 1.05367E-8 Infty
Alpha 2.00000 1.05367E-8 Infty
Gamma 2.00000 1.05367E-8 Infty

Optimization Summary
Optimization Technique Trust Region
Iterations 8
Function Calls 23
Log Likelihood -148.20614

Parameter Estimates
Parameter Estimate Standard
Error
t Value Approx
Pr > |t|
Theta 4.76980 0.62492 7.63 <.0001
Alpha 1.16363 0.58859 1.98 0.0509
Gamma 5.94081 1.05004 5.66 <.0001


You can specify a different set of initial values if estimates are available from fitting the distribution to similar data. For this example, the parameters of the Burr distribution can be initialized with the final parameter estimates of the Burr distribution that were obtained in the first example (shown in Figure 9.3). One of the ways in which you can specify the initial values is as follows:

/*------ Specifying initial values using INIT= option -------*/
proc hpseverity data=test_sev2 crit=aicc print=all;
   loss y / lt=threshold rc=limit;

   dist burr(init=(theta=4.62348 alpha=1.15706 gamma=6.41227));
   performance nthreads=2;
run;

The names of the parameters that are specified in the INIT option must match the parameter names in the definition of the distribution. The results obtained with these initial values are shown in Figure 9.7. These results indicate that new set of initial values causes the optimizer to reach the same solution with fewer iterations and function evaluations as compared to the default initialization.

Figure 9.7: Burr Model Optimization Summary for the Truncated and Censored Data

The HPSEVERITY Procedure
Burr Distribution

Optimization Summary
Optimization Technique Trust Region
Iterations 5
Function Calls 16
Log Likelihood -148.20614

Parameter Estimates
Parameter Estimate Standard
Error
t Value Approx
Pr > |t|
Theta 4.76980 0.62492 7.63 <.0001
Alpha 1.16363 0.58859 1.98 0.0509
Gamma 5.94081 1.05004 5.66 <.0001