Sample 25494: Textual analysis of word frequency
 |  |  |  |  |
These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.
This sample is from the SAS Sample Library. For additional information refer to SAS Help and Online Documentation.
/****************************************************************/
/* S A S S A M P L E L I B R A R Y */
/* */
/* NAME: WORDFREQ */
/* TITLE: Textual Analysis of Word Frequency */
/* PRODUCT: SAS */
/* SYSTEM: ALL */
/* KEYS: FREQUENCY COUNT */
/* PROCS: FREQ PLOT RANK SORT */
/* DATA: */
/* */
/* SUPPORT: UPDATE: */
/* REF: */
/* MISC: */
/* */
/****************************************************************/
DATA WORDS;
INPUT WORD : $ 16.@@;
LIST;
CARDS;
FOURSCORE AND SEVEN YEARS AGO OUR FATHERS BROUGHT
FORTH ON THIS CONTINENT A NEW NATION , CONCEIVED IN LIBERTY ,
AND DEDICATED TO THE PROPOSITION THAT ALL MEN ARE CREATED
EQUAL . NOW WE ARE ENGAGED IN A GREAT CIVIL WAR , TESTING
WHETHER THAT NATION , OR ANY NATION SO CONCEIVED AND SO
DEDICATED , CAN LONG ENDURE . WE ARE MET ON A GREAT BATTLEFIELD
OF THAT WAR . WE HAVE COME TO DEDICATE A PORTION OF
THAT FIELD , AS A FINAL RESTING PLACE FOR THOSE WHO HERE GAVE
THEIR LIVES THAT THAT NATION MIGHT LIVE . IT IS ALTOGETHER
FITTING AND PROPER THAT WE SHOULD DO THIS . BUT , IN A LARGER
SENSE , WE CANNOT DEDICATE WE CANNOT CONSECRATE WE
CANNOT HALLOW THIS GROUND . THE BRAVE MEN , LIVING AND
DEAD , WHO STRUGGLED HERE , HAVE CONSECRATED IT , FAR ABOVE
OUR POOR POWER TO ADD OR DETRACT . THE WORLD WILL LITTLE
NOTE , NOR LONG REMEMBER , WHAT WE SAY HERE , BUT IT CAN
NEVER FORGET WHAT THEY DID HERE . IT IS FOR US THE LIVING ,
RATHER , TO BE DEDICATED HERE TO THE UNFINISHED WORK WHICH
THEY WHO FOUGHT HERE HAVE THIS FAR SO NOBLY ADVANCED .
IT IS RATHER FOR US TO BE HERE DEDICATED TO THE GREAT TASK
REMAINING BEFORE US , THAT FROM THESE HONORED DEAD WE
TAKE INCREASED DEVOTION TO THAT CAUSE FOR WHICH THEY GAVE
THE LAST FULL MEASURE OF DEVOTION THAT WE HERE HIGHLY
RESOLVE THAT THESE DEAD SHALL NOT HAVE DIED IN VAIN THAT
THIS NATION UNDER GOD SHALL HAVE A NEW BIRTH OF FREEDOM
AND THAT GOVERNMENT OF THE PEOPLE , BY THE PEOPLE ,
FOR THE PEOPLE , SHALL NOT PERISH FROM THE EARTH .
;
RUN;
PROC FREQ;
TABLES WORD/NOPRINT OUT=COUNTS;
RUN;
DATA;
SET;
FILE PRINT;
IF WORD>='A';
N+1;
DROP N;
IF MOD(N,6)=1 THEN PUT;
PUT WORD $10. COUNT 5. +3 @;
RUN;
TITLE 'WORD FREQUENCIES OF LINCOLN''S GETTYSBURG ADDRESS';
PROC SORT;
BY DESCENDING COUNT;
RUN;
DATA;
FILE PRINT;
SET;
IF MOD(_N_,6)=1 THEN PUT;
PUT WORD $10. COUNT 5. +3 @;
RUN;
PROC RANK REVERSE;
VAR COUNT;
RANKS RANK;
RUN;
DATA;
SET;
LOGRANK=LOG(RANK);
LOGCOUNT=LOG(COUNT);
RUN;
PROC PLOT;
PLOT COUNT*RANK LOGCOUNT*LOGRANK;
RUN;
These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.
A 7 ABOVE 1 ADD 1 ADVANCED 1 AGO 1
ALL 1
ALTOGETHER 1 AND 6 ANY 1 ARE 3 AS 1
BATTLEFIEL 1
BE 2 BEFORE 1 BIRTH 1 BRAVE 1 BROUGHT 1
BUT 2
BY 1 CAN 2 CANNOT 3 CAUSE 1 CIVIL 1
COME 1
CONCEIVED 2 CONSECRATE 1 CONSECRATE 1 CONTINENT 1 CREATED 1
DEAD 3
DEDICATE 2 DEDICATED 4 DETRACT 1 DEVOTION 2 DID 1
DIED 1
DO 1 EARTH 1 ENDURE 1 ENGAGED 1 EQUAL 1
FAR 2
FATHERS 1 FIELD 1 FINAL 1 FITTING 1 FOR 5
FORGET 1
FORTH 1 FOUGHT 1 FOURSCORE 1 FREEDOM 1 FROM 2
FULL 1
GAVE 2 GOD 1 GOVERNMENT 1 GREAT 3 GROUND 1
HALLOW 1
HAVE 5 HERE 8 HIGHLY 1 HONORED 1 IN 4
INCREASED 1
IS 3 IT 5 LARGER 1 LAST 1 LIBERTY 1
LITTLE 1
LIVE 1 LIVES 1 LIVING 2 LONG 2 MEASURE 1
MEN 2
MET 1 MIGHT 1 NATION 5 NEVER 1 NEW 2
NOBLY 1
NOR 1 NOT 2 NOTE 1 NOW 1 OF 5
ON 2
OR 2 OUR 2 PEOPLE 3 PERISH 1 PLACE 1
POOR 1
PORTION 1 POWER 1 PROPER 1 PROPOSITIO 1 RATHER 2
REMAINING 1
REMEMBER 1 RESOLVE 1 RESTING 1 SAY 1 SENSE 1
SEVEN 1
SHALL 3 SHOULD 1 SO 3 STRUGGLED 1 TAKE 1
TASK 1
TESTING 1 THAT 13 THE 11 THEIR 1 THESE 2
THEY 3
THIS 5 THOSE 1 TO 8 UNDER 1 UNFINISHED 1
US 3
VAIN 1 WAR 2 WE 10 WHAT 2 WHETHER 1
WHICH 2
WHO 3 WILL 1 WORK 1 WORLD 1 YEARS 1
WORD FREQUENCIES OF LINCOLN'S GETTYSBURG ADDRESS
THAT 13 THE 11 WE 10 HERE 8 TO 8
A 7
AND 6 FOR 5 HAVE 5 IT 5 NATION 5
OF 5
THIS 5 DEDICATED 4 IN 4 ARE 3 CANNOT 3
DEAD 3
GREAT 3 IS 3 PEOPLE 3 SHALL 3 SO 3
THEY 3
US 3 WHO 3 BE 2 BUT 2 CAN 2
CONCEIVED 2
DEDICATE 2 DEVOTION 2 FAR 2 FROM 2 GAVE 2
LIVING 2
LONG 2 MEN 2 NEW 2 NOT 2 ON 2
OR 2
OUR 2 RATHER 2 THESE 2 WAR 2 WHAT 2
WHICH 2
ABOVE 1 ADD 1 ADVANCED 1 AGO 1 ALL 1
ALTOGETHER 1
ANY 1 AS 1 BATTLEFIEL 1 BEFORE 1 BIRTH 1
BRAVE 1
BROUGHT 1 BY 1 CAUSE 1 CIVIL 1 COME 1
CONSECRATE 1
CONSECRATE 1 CONTINENT 1 CREATED 1 DETRACT 1 DID 1
DIED 1
DO 1 EARTH 1 ENDURE 1 ENGAGED 1 EQUAL 1
FATHERS 1
FIELD 1 FINAL 1 FITTING 1 FORGET 1 FORTH 1
FOUGHT 1
FOURSCORE 1 FREEDOM 1 FULL 1 GOD 1 GOVERNMENT 1
GROUND 1
HALLOW 1 HIGHLY 1 HONORED 1 INCREASED 1 LARGER 1
LAST 1
LIBERTY 1 LITTLE 1 LIVE 1 LIVES 1 MEASURE 1
MET 1
MIGHT 1 NEVER 1 NOBLY 1 NOR 1 NOTE 1
NOW 1
PERISH 1 PLACE 1 POOR 1 PORTION 1 POWER 1
PROPER 1
PROPOSITIO 1 REMAINING 1 REMEMBER 1 RESOLVE 1 RESTING 1
SAY 1
SENSE 1 SEVEN 1 SHOULD 1 STRUGGLED 1 TAKE 1
TASK 1
TESTING 1 THEIR 1 THOSE 1 UNDER 1 UNFINISHED 1
VAIN 1
WHETHER 1 WILL 1 WORK 1 WORLD 1 YEARS 1
WORD FREQUENCIES OF LINCOLN'S GETTYSBURG ADDRESS
Plot of COUNT*RANK. Legend: A = 1 obs, B = 2 obs, etc.
|
|
13 + A
|
|
12 +
|
|
11 + A
|
|
10 + A
F |
r |
e 9 +
q |
u |
e 8 + B
n |
c |
y 7 + A
|
C |
o 6 + A
u |
n |
t 5 + F
|
|
4 + B
|
|
3 + K
|
|
2 + V
|
|
1 + Z
|
---+--------------+--------------+--------------+--------------+--------------+--
0 20 40 60 80 100
Rank for Variable COUNT
NOTE: 63 obs hidden.
WORD FREQUENCIES OF LINCOLN'S GETTYSBURG ADDRESS
Plot of LOGCOUNT*LOGRANK. Legend: A = 1 obs, B = 2 obs, etc.
LOGCOUNT |
3.0 +
|
|
|
|
|
| A
2.5 +
| A
|
| A
|
|
| B
2.0 +
| A
|
| A
|
| F
|
1.5 +
|
| B
|
|
|
| K
1.0 +
|
|
|
| V
|
|
0.5 +
|
|
|
|
|
|
0.0 + Z
|
---+-------------+-------------+-------------+-------------+-------------+--
0 1 2 3 4 5
LOGRANK
NOTE: 63 obs hidden.
This example implements textual analysis of word frequency.
| Type: | Sample |
| Topic: | SAS Reference ==> Procedures ==> FREQ Analytics ==> Nonparametric Analysis Analytics ==> Longitudinal Analysis Analytics ==> Descriptive Statistics Analytics ==> Exact Methods SAS Reference ==> Procedures ==> PLOT Analytics ==> Categorical Data Analysis |
| Date Modified: | 2005-11-04 03:02:52 |
| Date Created: | 2005-05-23 13:57:35 |
Operating System and Release Information
| Product Family | Product | Host | SAS Release | |
| Starting | Ending | |||
| SAS System | Base SAS | All | n/a | n/a |