35230 - Shorten character variables to their minimum required length

Sample 35230: Shorten character variables to their minimum required length

This sample code below finds the maximum length of each character variable in the data set, writes a DATA step that issues new LENGTH and FORMAT statements for each character variable, and then reads the data. The goal is to make the data set as small as possible without using compression.

The first DATA step counts the number of character variables in the data set and stores this value in a macro variable. The next step loops through all of the data looking for the maximum number of characters in each variable. When all of the rows have been checked, a LENGTH and FORMAT statements are generated for each variable and saved in macro variables. In the final DATA step, the macro loop pushes the LENGTH and FORMAT statements out to the DATA step ahead of the SET statement so that the length and format are changed.

To avoid the following WARNING, add this OPTIONS statement to the top of your program: OPTIONS VARLENCHK=NOWARN;

WARNING: Multiple lengths were specified for the variable bb by input data set(s). This can cause truncation of data.

Note that a new data set is created rather than overwriting the original. If you prefer to replace the original, remove the underscore from the final DATA statement.

%macro change(dsn);                                         

data _null_;                                               

  set &dsn;                                                

  array qqq(*) _character_;                                

  call symput('siz',put(dim(qqq),5.-L));                   

  stop;                                                    

run;

data _null_;                                               

  set &dsn end=done;                                       

  array qqq(&siz) _character_;                             

  array www(&siz.);                                        

  if _n_=1 then do i= 1 to dim(www);                       

    www(i)=0;                                              

  end;                                                     

  do i = 1 to &siz.;                                       

    www(i)=max(www(i),length(qqq(i)));                     

  end;                                                     

  retain _all_;                                            

  if done then do;                                         

    do i = 1 to &siz.;                                     

      length vvv $50;                                      

      vvv=catx(' ','length',vname(qqq(i)),'$',www(i),';'); 

      fff=catx(' ','format ',vname(qqq(i))||' '||          

          compress('$'||put(www(i),3.)||'.;'),' ');        

      call symput('lll'||put(i,3.-L),vvv) ;                

      call symput('fff'||put(i,3.-L),fff) ;                

    end;                                                   

  end;                                                     

run;

data &dsn._;                                               

  %do i = 1 %to &siz.;                                     

    &&lll&i                                                

    &&fff&i                                                

  %end;                                                    

  set &dsn;                                                

run;

%mend;                                                      

data test;

infile datalines truncover;

input aa $3. bb :$15.;

datalines;

123 4567

;

%change(work.test)

These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.

Type:	Sample
Topic:	SAS Reference ==> DATA Step

Date Modified:	2020-11-17 14:18:25
Date Created:	2009-03-20 17:30:18

Product Family	Product	Host	SAS Release
Product Family	Product	Host	Starting	Ending
SAS System	Base SAS	Linux on Itanium
		Linux
		Linux for x64
		IRIX
		HP-UX IPF
		HP-UX
		AIX
		ABI+ for Intel Architecture
		64-bit Enabled Solaris
		64-bit Enabled HP-UX
		64-bit Enabled AIX
		Windows Vista
		Windows Millennium Edition (Me)
		Microsoft Windows XP Professional
		Microsoft Windows Server 2003 Standard Edition
		Microsoft Windows Server 2003 Enterprise Edition
		Microsoft Windows Server 2003 Datacenter Edition
		Microsoft Windows NT Workstation
		Microsoft Windows 2000 Professional
		Microsoft Windows 2000 Server
		Microsoft Windows 2000 Datacenter Server
		Microsoft Windows 2000 Advanced Server
		Microsoft Windows 95/98
		OS/2
		Microsoft® Windows® for x64
		Microsoft Windows XP 64-bit Edition
		Microsoft Windows Server 2003 Enterprise 64-bit Edition
		Microsoft Windows Server 2003 Datacenter 64-bit Edition
		Microsoft® Windows® for 64-Bit Itanium-based Systems
		OpenVMS VAX
		z/OS
		OpenVMS Alpha
		OpenVMS on HP Integrity
		Solaris
		Solaris for x64
		Tru64 UNIX

Support

Sample 35230: Shorten character variables to their minimum required length

Operating System and Release Information