Using the Optimum Range for Best Output Results

Suppose you see the following warnings in your SAS log:

   libname kgs2 sasehavr "%sysget(HAVER_DATA)"
           start= 19550101 end=19600105
           keep="FCSEED, FCSEEI, FCSEEM, BGSX, BGSM, FXDUSBC"
           group="I01, F56, M02, R30"
           source="JPM,CEN,OMB" ;

   NOTE: Libref KGS2 was successfully assigned as follows: 
         Engine:        HAVERDLX 
         Physical Name: C:\haver
         
   data kgse9;
      set kgs2.haver;
   NOTE: Defaulting to MONTHLY frequency.
   WARNING: Start date (19550101) is not a valid date. 
            Engine is ignoring your start date and using 
            default. Setting the default Haver start date to 7001.
   WARNING: End date (19600105) is not a valid date. 
            Engine is ignoring your end date and using 
            default. Setting the default Haver end date to 10103.
   run;

   NOTE: There were 375 observations read from the data set KGS2.HAVER.
   NOTE: The data set WORK.KGSE9 has 375 observations and 4 variables.

The important diagnostic to note here is the warning message which tells you that the data starts in January of 1970 (HAVER date 7001), and ends in March, 2001 (HAVER date 10103). Since the specified range falls outside the range of data, no observations are in range so the engine uses the default range stated in the warning messages. Changing the START= and END= options to overlap the results in data spanning from JAN1970 to MAR2001. To view the entire range of selected data, remove the START= and END= options from your LIBNAME statement:

   libname kgs sasehavr "%sysget(HAVER_DATA)"
           keep="FCSEED, FCSEEI, FCSEEM, BGSX, BGSM, FXDUSBC"
           group="I01, F56, M02, R30"
           source="JPM,CEN,OMB" ;

   NOTE: Libref KGS was successfully assigned as follows: 
         Engine:        HAVERDLX 
         Physical Name: C:\haver
         
   data kgse5;
      set kgs.haver;
   NOTE: Defaulting to MONTHLY frequency.
      run;

   NOTE: There were 375 observations read from the data set KGS.HAVER.
   NOTE: The data set WORK.KGSE5 has 375 observations and 4 variables.

Using a Valid Range of Data With START= and END= Options

In this example, an error about an invalid range is issued:

libname lib1 sasehavr "%sysget(HAVER_DATA)"  freq=Weekly
   start=20060301 end=20060531;
NOTE: Libref LIB1 was successfully assigned as follows: 
      Engine:        HAVERDLX 
      Physical Name: C:\haver
libname lib2 "\\dntsrc\usrtmp\saskff" ;
NOTE: Libref LIB2 was successfully assigned as follows: 
      Engine:        V9 
      Physical Name: \\dntsrc\usrtmp\saskff         
data lib2.wweek;
   set lib1.intwkly;
ERROR: No observations found inside RANGE.
       The valid range for HAVER dates is (610104-1050318).
ERROR: No observations found in specified range.
            keep date m11: ;
         run;

WARNING: The variable date in the DROP, KEEP, or RENAME list 
         has never been referenced.
WARNING: The variable m11: in the DROP, KEEP, or RENAME list 
         has never been referenced.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set LIB2.WWEEK may be incomplete.  
         When this step was stopped there were 0 
         observations and 0 variables.
WARNING: Data set LIB2.WWEEK was not replaced because this step was stopped.

In the preceding example, the important diagnostic message is the first error statement which tells you that the RANGE of HAVER dates is invalid for the specified frequency. A valid range is one that overlaps the dates (610104-1050318). Removing the range altogether will cause the engine to output the entire range of data.

libname lib1 sasehavr "%sysget(HAVER_DATA)"  freq=Weekly;
  
NOTE: Libref LIB1 was successfully assigned as follows: 
      Engine:        HAVERDLX 
      Physical Name: C:\haver

libname lib2 "\\dntsrc\usrtmp\saskff" ;
NOTE: Libref LIB2 was successfully assigned as follows: 
      Engine:        V9 
      Physical Name: \\dntsrc\usrtmp\saskff
         
data lib2.wweek;
   set lib1.intwkly;
   keep date m11: ;
run;

NOTE: There were 2307 observations read from the data set LIB1.INTWKLY.
NOTE: The data set LIB2.WWEEK has 2307 observations and 35 variables.

When giving a range of dates, since the START=, END= options give day-based dates, its important to use dates that correspond to the FREQ= option, especially with weekly frequencies, such as week.1-week.7. Since the FREQ=week.4 selects weeks that begin on Wednesday, the start and end dates need to be specified as Wednesday dates.

libname lib1 sasehavr "%sysget(HAVER_DATA)"  freq=Week.4
   start=20050302 end=20050309;
NOTE: Libref LIB1 was successfully assigned as follows: 
      Engine:        HAVERDLX 
      Physical Name: \\tappan\crsp1\haver
title2 'Weekly dataset with freq=week.4 range is small';
libname lib2 "\\dntsrc\usrtmp\saskff" ;
NOTE: Libref LIB2 was successfully assigned as follows: 
      Engine:        V9 
      Physical Name: \\dntsrc\usrtmp\saskff
         
data lib2.wweek;
   set lib1.intwkly;
   keep date m11: ;
run;

NOTE: There were 2 observations read from the data set LIB1.INTWKLY.
NOTE: The data set LIB2.WWEEK has 2 observations and 25 variables.

Giving bad dates (i.e., Tuesday dates) for a Wednesday FREQ=week.4 will result in the following error.

ERROR: Fatal error in GetDate routine. 
       Remove the range statement or change the START= date to 
       be consistent with the freq=option.
ERROR: No observations found in specified range.

Aggregating to Quarterly Frequency Using the FORCE= FREQ Option

In the next example, 6 time series are selected by the KEEP=option, but their frequencies are annual, monthly, quarterly, so when the FREQ= weekly and FORCE= freq options are used, a diagnostic appears in the log stating that the engine is forcing the frequency to QUARTERLY for better date alignment of observations. The first selected variable is BALO which is a quarterly time series, which causes the default choice of freq to be quarterly:

title1 '***HAVKWC.SAS: KEEP= option tests with wildcards***';

%setup( ets );

 /*----------------*/
 /* Wildcard: *    */
 /*----------------*/

title2 "keep=B*, G*, I*";
title3 "6 valid variables are: BALO BGSM BGSX BPBCA G IUM";
libname lib1 sasehavr 'C:\haver\' keep="B*, G*, I*"
   freq=weekly force=freq;
NOTE: Libref LIB1 was successfully assigned as follows: 
      Engine:        HAVERDLX 
      Physical Name: C:\haver\

data wc;
   set lib1.haver;
WARNING: Earliest Start Date in DLX Database matches QUARTERLY frequency
         better than the specified WEEKLY frequency.
         Engine is forcing the frequency to QUARTERLY for better date
         alignment of observations.
run;

NOTE: There were 221 observations read from the data set LIB1.HAVER.
NOTE: The data set WORK.WC has 221 observations and 7 variables.

Note that the time series IUM is an annual frequency, so the attempt to convert to a quarterly frequency produces all missing values in the output range because aggregation produces only missing values when going from a lower frequency to a higher (forced) frequency.