Seria Wiedza Biznesowa szkolenie

This course, which was completely rewritten to be compatible with the third edition of the book Cody's Data Cleaning Techniques Using SAS, will help greatly speed up the process of detecting and correcting errors in both character and numeric data. In addition, there are sections on standardizing data and using Perl regular expressions to ensure that character values conform to a specific pattern (such as ZIP codes, phone numbers, and email addresses).

Although the course concentrates on methods of identifying data errors, it also teaches some programming techniques that might be new to you. For example, by using some of the latest SAS functions, you can convert a phone number in just about any form into a standard form, in only two SAS statements!

The course teaches several methods of detecting errors in numeric data including range checking as well as several methods of automatic outlier detection. There are chapters devoted to data that involves multiple observations per subject, SAS dates, and projects that include multiple data sets. The class closes with a demonstration of an innovative process that leverages integrity constraints and audit trails to detect and programmatically clean dirty data before it even gets into your analysis data set.

All students taking this class are presented with either a printed version or PDF version of the new Data Cleaning book and are given access to dozens of macros that will greatly speed up the laborious process of cleaning your data.

• check character and numeric variables for invalid values
• identify and count missing values for numeric and character variables
• work with and manipulate dates
• work with data sets that have multiple observations per subject
• identify inconsistencies among multiple files
• compare two data sets with and without an ID variable
• use integrity constraints and audit trails to weed out data errors.

Kto powinien uczestniczyć

Before attending this course, participants should have completed the Podstawy programowania w SAS 4GL (dla nieprogramistów) course. Completion of the Łączenie tabel przy użyciu SAS 4GL (dla nieprogramistów) course or a minimum of one year of SAS programming experience is also recommended.

To szkolenie wykorzystuje oprogramowanie Base SAS, SAS/STAT  

• using the FREQ procedure to list values
• using the DATA step to check for invalid values
• using the PRINT procedure to list invalid values
• using formats to check for invalid values

• an introduction to regular expressions
• a list of some common regular expressions
• repetition operators
• the PRXMATCH function
• examples of using regular expressions to test for valid patterns
• creating macros to test regular expressions

• using formats to standardize data
• creating a format from a data set
• using TRANWRD to standardize addresses

• using the UNIVARIATE procedure to look for outliers
• describing the ODS SELECT statement
• a program to list the 10 highest and lowest values
• describing how macros work
• demonstrating a conditional SET statement
• listing the top and bottom n percent of data values
• describing the RANK procedure
• listing out-of-range values using a DATA step
• listing invalid data values
• a program to summarize data errors

• looking for outliers using the mean and standard deviation
• describing and using trimmed statistics
• using PROC UNIVARIATE to compute trimmed statistics
• demonstrating the Auto_Outliers macro
• detecting outliers based on the interquartile range

• identifying missing values
• counting missing and nonmissing values

• checking dates
• working with dates in nonstandard form

• detecting duplicates
• identifying duplicates and verifying n observations per subject

• checking for an ID in each of two files
• checking for an ID in each of three files

• conducting a simple comparison
• double entry verification

• making simple corrections
• introduction to integrity constraints and audit trails