FOCUS AREAS

SAS/CONNECT - Tips and Tricks

Template for Signing on Multiple Times to the Same Machine

Basic SAS/CONNECT Parallel Processing Template - SMP Machine

Basic SAS/CONNECT Parallel Processing Template - Remote Machines

Get Path of WORK Library for SAS/CONNECT Sessions

Using LOG=/OUTPUT= Options on Async RSUBMIT

Piping Between Data Steps

Piping Between Data Step and Proc Sort on SMP Machine

Piping Between Data Step and Proc Sort Across Remote Machines

Multi-Processing with Raw Data Files and Merging

Piping with Raw Data Files and Merging

Piping Between Data Step, Sort, and Merge


Template for Signing on Multiple Times to the Same Machine

This tip shows you how to signon multiple times to the same remote machine. This is useful when the remote machine is an SMP machine and you can benefit from having more than one remote process running there.

You will need to define an appropriate script file if using a script to signon. If signing on to a spawner without a script file, don't worry with this statement.


filename rlink 'myscript.scr';

Because the remote= values must be unique between connections, you can use unique macro variables and just assign all of them to the same machine name. This will create three seperate connections to three SAS sessions running on rem.addr.of.node (which you will need to replace with the host name or ip address of your machine.

%let conn1=rem.addr.of.node;
%let conn2=rem.addr.of.node;
%let conn3=rem.addr.of.node;
signon conn1;
signon conn2;
signon conn3;

Basic SAS/CONNECT Parallel Processing Template - SMP Machine

The following pseudo code sets up the global SASCMD option so that N number of SAS sessions can be spawned on the same multi-processor machine. Each SIGNON statement spawns a new SAS session. The %SYSLPUT statement can be used to set macro variables in the "remote" SAS sessions. The RSUBMIT statements must have the WAIT=NO option in order to cause the RSUBMITs to execute asynchronously. In addition, the LOG=/OUTPUT= options are used in this example to manage the output and results from each of the async tasks.

options sascmd="sas";

signon task1;
%syslput remvar1=somevalue;

rsubmit task1 wait=no log="task1.log" output="task1.lst";
   /* send a task to be executed by TASK1 SAS session */
   /* may make use of REMVAR1 macro variable in this  */
   /* SAS code */
endrsubmit;


signon task2;
%syslput remvar2=somevalue;

rsubmit task2 wait=no log="task2.log" output="task2.lst";
   /* send a task to be executed by TASK2 SAS session */
   /* may make use of REMVAR2 macro variable in this  */
   /* SAS code */
endrsubmit;

signon taskn;
%syslput remvarn=somevalue;

rsubmit taskn wait=no log="taskn.log" output="taskn.lst";
   /* send a task to be executed by TASKn SAS session */
   /* may make use of REMVARn macro variable in this  */
   /* SAS code */
endrsubmit;


waitfor _all_ task1 task2 ... taskn;

/* do some further local processing */

signoff task1;
signoff task2;
signoff taskn;

Basic SAS/CONNECT Parallel Processing Template - Remote Machines

The following pseudo code assumes that you have several Windows workstations available for parallel processing. This scenario also assumes that a spawner is running on each of the Windows boxes waiting for incoming requests for connections. Each SIGNON statement creates a connection and a new SAS session on the specified remote Windows machine. The WIN1 through WINn machine names must be defined either in the SERVICES file on the client machine or in a name server on the network. The %SYSLPUT statement can be used to set macro variables in the remote SAS sessions. The RSUBMIT statements must have the WAIT=NO option in order to cause the RSUBMITs to execute asynchronously. In addition, the LOG=/OUTPUT= options are used in this example to manage the output and results from each of the async tasks.

signon win1 userid=_prompt_;
%syslput remvar1=somevalue;

rsubmit win1 wait=no log="win1.log" output="win1.lst";
   /* send a task to be executed by SAS session on */
   /* WIN1 machine; may make use of REMVAR1 macro  */
   /* variable in this SAS code                    */
endrsubmit;


signon win2 userid=_prompt_;
%syslput remvar2=somevalue;

rsubmit win2 wait=no log="win2.log" output="win2.lst";
   /* send a task to be executed by SAS session on */
   /* WIN2 machine; may make use of REMVAR2 macro  */
   /* variable in this SAS code                    */
endrsubmit;

signon winn userid=_prompt_;
%syslput remvarn=somevalue;

rsubmit winn wait=no log="winn.log" output="winn.lst";
   /* send a task to be executed by SAS session on */
   /* WINn machine; may make use of REMVARn macro  */
   /* variable in this SAS code                    */
endrsubmit;


waitfor _all_ win1 win2 ... winn;

/* do some further local processing */

signoff win1;
signoff win2;
signoff winn;

Get Path of WORK Library for CONNECT Sessions

This program uses SAS/CONNECT to create two SAS sessions on an SMP machine to perform two independent sorts in parallel. It also makes use of the %sysrput statement and pathname function to return the physical path of the spawned sessions work libraries back to the parent session so that the sorted data sets can then be merged in the parent SAS session.

options autosignon=yes sascmd="sas";

/* create TASK1 SAS session to sort the SALES data */
rsubmit task1 wait=no sysrputsync=yes;
   proc sort data=sales;
      by date;
   run;
   /* get TASK1's WORK library path in parent session */
   %sysrput pathtask1=%sysfunc(pathname(work));
endrsubmit;

/* create TASK2 SAS session to sort the GOALS data */
rsubmit task2 wait=no sysrputsync=yes;
   proc sort data=goals;
      by date;
   run;
   /* get TASK2's WORK library path in parent session */
   %sysrput pathtask2=%sysfunc(pathname(work));
endrsubmit;

/* wait until both sorts have finished */
waitfor _all_ task1 task2;

/* now we can define a libref to the sorted data sets and */
/* merge them inthe parent session                        */
libname worktsk1 "&pathtask1";
libname worktsk2 "&pathtask2";
data combine;
   set worktsk1.sales
       worktsk2.goals;
   by date;
run;

signoff task1;
signoff task2;

Using LOG=/OUTPUT= Options on Async RSUBMIT

This example shows the use of the LOG=/OUTPUT= options on async rsubmits. Two external files are created in the specified location; one for the log lines and one for the output lines. (If you are connecting to remote machines, the files are created on the local or client machine.) The desire is to have all log and output lines go to these two files from multiple rsubmits. A fileref is used as the value for these options so that the MOD option can be used on the filename statement to cause lines to be appended to the same file instead of creating a brand new file on each subsequent RSUBMIT.

NOTE: If you are running multiple connections (you've signed on to multiple sessions), you may want to define a pair of log and output files for each connection.

options autosignon=yes sascmd="sas";
filename mylog 'a/location/for/log/lines' mod;
filename myout 'a/location/for/output/lines' mod;

rsubmit task1 wait=no log=mylog output=myout;
  /* some asynchronous task*/
endrsubmit;

/* some local processing */

rsubmit task1 wait=no log=mylog output=myout;
  /* another asynchronous task*/
endrsubmit;

waitfor _all_ task1;

At this point, the files referenced by MYLOG and MYOUT will contain all of the log and output lines from the RSUBMITs to TASK1.

Piping Between Data Steps

This example shows the new Version 9 SAS/CONNECT piping capability from one SAS data step to another. In this example, SAS/CONNECT is used to spawn two SAS sessions: session P1 and session P2. Session P1 defines a libref OUTLIB which uses the pipe engine to pipe the output of the OUTLIB.INTERMEDIATE data set as input to the data step running in session P2. The OUTLIB.FINAL data set that gets created by session P2 is written to disk.

/* start session P1 and execute first data step */
/* asynchronously to a pipe */
signon p1 sascmd='sas';
rsubmit p1  wait=no;
   libname outLib sasesock ":pipe1";
   data outLib.Intermediate;
      do i=1 to 5;
          put 'Writing row ' i; output;
      end;
   run;
endrsubmit;

/* start session P2 and execute second data step which */
/* gets its input from a pipe */
signon p2 sascmd='sas';
rsubmit p2  wait=no;
   libname inLib sasesock ":pipe1";
   libname outLib "d:\temp";

   data outLib.Final;
      set inLib.Intermediate;
      do j=1 to 5;
           put 'Adding data ' j;
           n2 = j*2; output;
      end;
   run;
endrsubmit;

waitfor _all_ p1 p2;

signoff p1;
signoff p2;

Piping Between Data Step and Proc Sort on SMP Machine

This example uses a data step to hash a large data file into four subsets. Once the data has been divided into four subsets, four sorts are run on the data in parallel. Then a final data step is used to merge the sorted subsets into the final result. SAS/CONNECT is used to run a total of five SAS sessions:

The power of this example is that piping can be used to overlap all of these tasks. The client or parent session spawns the four SAS sessions to run proc sort. Because these sessions are all running asynchronously, the parent session can then immediately start the first data step to subset the data and pipe the subsets to the four running sorts. The sorts then turn around and pipe their output back to the parent session where it serves as input to the final data step to do the merge.

/* set up the global option to cause the spawned sessions */
/* to be started with the same command that was used to   */
/* invoke the client/parent SAS session.                  */
options sascmd="!sascmd";

libname out1 sasesock ":pipe1";
libname out2 sasesock ":pipe2";
libname out3 sasesock ":pipe3";
libname out4 sasesock ":pipe4";
libname in1 sasesock ":pipe5";
libname in2 sasesock ":pipe6";
libname in3 sasesock ":pipe7";
libname in4 sasesock ":pipe8";

signon p1;
signon p2;
signon p3;
signon p4;

rsubmit p1 wait=no;
 libname in1 sasesock ":pipe1";
 libname out1 sasesock ":pipe5";
 proc sort data=in1.test out=out1.test;
 by k;
 run;
endrsubmit;

rsubmit p2 wait=no;
 libname in2 sasesock ":pipe2";
 libname out2 sasesock ":pipe6";
 proc sort data=in2.test out=out2.test;
 by k;
 run;
endrsubmit;

rsubmit p3 wait=no;
 libname in3 sasesock ":pipe3";
 libname out3 sasesock ":pipe7";
 proc sort data=in3.test out=out3.test;
 by k;
 run;
endrsubmit;

rsubmit p4 wait=no;
 libname in4 sasesock ":pipe4";
 libname out4 sasesock ":pipe8";
 proc sort data=in4.test out=out4.test;
 by k;
 run;
endrsubmit;


data out1.test out2.test out3.test out4.test;
 drop i j;
 length x $ 80;
 x = repeat(" ", 79);
 j = 0;
 do i=1 to 100000000;
   k=ranuni(1);
   if ( j = 0 ) then
     output out1.test;
   else if ( j = 1 ) then
     output out2.test;
   else if ( j = 2 ) then
     output out3.test;
   else output out4.test;
   j = j + 1;
   if ( j = 4 ) then j = 0;
  end;
run;

data bar;
   set in1.test; by k; output;
   set in2.test; by k; output;
   set in3.test; by k; output;
   set in4.test; by k; output;
   run;

waitfor _ALL_ p1 p2 p3 p4;

signoff p1;
signoff p2;
signoff p3;
signoff p4;

Piping Between Data Step and Proc Sort Across Remote Machines

Just as in the previous example, this example also executes a data step to hash a large data file into four subsets. Once the data has been divided into four subsets, four sorts are run on the data in parallel. However, in this example the sorts are executed on four remote machines. Then a final data step is used to merge the sorted subsets into the final result. SAS/CONNECT is used to run a total of six SAS sessions:

The power of this example is that piping can be used to overlap all of these tasks across remote machines when you do not have the horsepower of an SMP machine available. The client or parent session spawns the four SAS sessions to run proc sort. Because these sessions are all running asynchronously, the parent session can then immediately start the first data step to subset the data and pipe the subsets to the four running sorts. The sorts then turn around and pipe their output back to the parent session where it serves as input to the final data step to do the merge.

filename rlink '!sasroot\connect\saslink\tcpwin.scr';
%let p1=host1.a.b.com;
%let p2=host2.a.b.com;
%let p3=host3.a.b.com;
%let p4=host4.a.b.com;

libname out1 sasesock ":pipe1";
libname out2 sasesock ":pipe2";
libname out3 sasesock ":pipe3";
libname out4 sasesock ":pipe4";

/* signon to four remote hosts; each will run a sort */
signon p1;
signon p2;
signon p3;
signon p4;

/* p1-p4 gets their input from data step running in  */
/* client/parent session                             */
rsubmit p1 wait=no;
 libname in1 sasesock "parent.b.c.com:pipe1";
 libname out1 sasesock ":pipe5";
 proc sort data=in1.foo out=out1.foo;
 by k;
 run;
endrsubmit;

rsubmit p2 wait=no;
 libname in2 sasesock "parent.b.c.com:pipe2";
 libname out2 sasesock ":pipe6";
 proc sort data=in2.foo out=out2.foo;
 by k;
 run;
endrsubmit;

rsubmit p3 wait=no;
 libname in3 sasesock "parent.b.c.com:pipe3";
 libname out3 sasesock ":pipe7";
 proc sort data=in3.foo out=out3.foo;
 by k;
 run;
endrsubmit;

rsubmit p4 wait=no;
 libname in4 sasesock "parent.b.c.com:pipe4";
 libname out4 sasesock ":pipe8";
 proc sort data=in4.foo out=out4.foo;
 by k;
 run;
endrsubmit;

/* hash original data set and pipe subsets to each   */
/* of the four sorts                                 */
data out1.foo out2.foo out3.foo out4.foo;
 drop i j;
 length x $ 80;
 x = repeat(" ", 79);
 j = 0;
 do i=1 to 50000000;
   k=ranuni(1);
   if ( j = 0 ) then
     output out1.foo;
   else if ( j = 1 ) then
     output out2.foo;
   else if ( j = 2 ) then
     output out3.foo;
   else output out4.foo;
   j = j + 1;
   if ( j = 4 ) then j = 0;
  end;
run;

/* create a session to do the final merge */
signon localM sascmd='!sascmd';
rsubmit localM wait=no;
 libname in1 sasesock "host1.a.b.com:pipe5";
 libname in2 sasesock "host2.a.b.com:pipe6";
 libname in3 sasesock "host3.a.b.com:pipe7";
 libname in4 sasesock "host4.a.b.com:pipe8";
 data bar;
    set in1.foo; by k; output;
    set in2.foo; by k; output;
    set in3.foo; by k; output;
    set in4.foo; by k; output;
run;
endrsubmit;

waitfor _ALL_ p1 p2 p3 p4 localM;

signoff p1;
signoff p2;
signoff p3;
signoff p4;
signoff localM;

Multi-Processing with Raw Data Files and Merging

This example two raw data files must be input. They are then divided into four data sets, each representing a quarter of the year. The final step is to merge the two data sets that apply to each quarter. The parallel processing capabilities of SAS/CONNECT are used to perform all of the independent tasks in parallel.

The first raw data file contains the projected sales goals for the four quarters of the year. One process reads this data and divides it into four data sets, one for each quarter of the year. The second raw data file contains the actual sales for the four quarters of the year. A second process simultaneously reads this data and also divides it into four data sets, one for each quarter of the year. Once both of these tasks are complete, the next step of the application is to merge the projected and actual sales for each of the four quarters. A session is created for each of the quarters and all four sessions execute in parallel to merge the data sets for a particular quarter.

/* create the sessions that will be needed to execute in */
/* parallel - one for each of the 4 quarters of the year */
signon task1 sascmd="!sascmd" inheritlib=(Work=locwork);
signon task2 sascmd="!sascmd" inheritlib=(Work=locwork);
signon task3 sascmd="!sascmd" inheritlib=(Work=locwork);
signon task4 sascmd="!sascmd" inheritlib=(Work=locwork);

/* use the first session to input the first raw file and    */
/* divide the records into 4 datasets, one for each quarter */
rsubmit task1 wait=no;
data locwork.qtr1_sales locwork.qtr2_sales
     locwork.qtr3_sales locwork.qtr4_sales;
  infile '/DATA01/mpconnect/jmk4/sales.txt' dlm=':';
  input EmpID $ 12. Qtr1 Qtr2 Qtr3 Qtr4;
  drop Qtr1 Qtr2 Qtr3 Qtr4 Qtr;
  array contrib{4} Qtr1-Qtr4;
  do Qtr=1 to 4;
    Amount_sold = Contrib{Qtr};
    if Qtr=1 then output locwork.qtr1_sales;
    else if Qtr=2 then output locwork.qtr2_sales;
    else if Qtr=3 then output locwork.qtr3_sales;
    else if Qtr=4 then output locwork.qtr4_sales;
  end;
run;
endrsubmit;

/* use the second session to input the second raw file and  */
/* divide the records into 4 datasets, one for each quarter */
/* NOTE: task1 and task2 will run in parallel               */
rsubmit task2 wait=no;
data locwork.qtr1_goals locwork.qtr2_goals
     locwork.qtr3_goals locwork.qtr4_goals;
  infile '/DATA01/mpconnect/jmk4/goals.txt' dlm=':';
  input EmpID $ 12. Qtr1 Qtr2 Qtr3 Qtr4;
  drop Qtr1 Qtr2 Qtr3 Qtr4 Qtr;
  array contrib{4} Qtr1-Qtr4;
  do Qtr=1 to 4;
    Amount_goal = Contrib{Qtr};
    if Qtr=1 then output locwork.qtr1_goals;
    else if Qtr=2 then output locwork.qtr2_goals;
    else if Qtr=3 then output locwork.qtr3_goals;
    else if Qtr=4 then output locwork.qtr4_goals;
  end;
run;
endrsubmit;

/* wait for above two steps to complete */
waitfor _ALL_ task1 task2;

/* execute all 4 sessions in parallel to each do a merge of */
/* the two data sets for their respective quarters.         */
rsubmit task1 wait=no;
data locwork.qtr1_comp;
  merge locwork.qtr1_sales locwork.qtr1_goals;
  by EmpID;
run;
endrsubmit;

rsubmit task2 wait=no;
data locwork.qtr2_comp;
  merge locwork.qtr2_sales locwork.qtr2_goals;
  by EmpID;
run;
endrsubmit;

rsubmit task3 wait=no;
data locwork.qtr3_comp;
  merge locwork.qtr3_sales locwork.qtr3_goals;
  by EmpID;
run;
endrsubmit;

rsubmit task4 wait=no;
data locwork.qtr4_comp;
  merge locwork.qtr4_sales locwork.qtr4_goals;
  by EmpID;
run;
endrsubmit;

waitfor _ALL_ task1 task2 task3 task4;

Piping with Raw Data Files and Merging

This example builds on the previous example above by using the piping feature to eliminate the write of temporary files on disk. Instead, the data is piped from a data step, performing subsetting, directly into a proc merge. In this application two raw data files must be input. They are then divided into four data sets, each representing a quarter of the year. The final step is to merge the two data sets that apply to each quarter.

The first raw data file contains the projected sales goals for the four quarters of the year. One process reads this data and divides it into four subsets, one for each quarter of the year. The second raw data file contains the actual sales for the four quarters of the year. A second process simultaneously reads this data and also divides it into four subsets, one for each quarter of the year. The next step of the application is to merge the projected and actual sales for each of the four quarters. Since the intermediate quarterly subsets of the data do not need to be stored permanently, piping is used to allow the two data steps executing in parallel to send their output to one of 4 pipes (one for each quarter) and pipe the quarterly data into the appropriate session to handle the merge for that quarter. By using piping, we:

/* create the sessions that will execute in parallel  */
/* 2 for the 2 parallel data steps and 4 more for the */
/* merging based on the quarter.                      */
signon task1 sascmd="!sascmd";
signon task2 sascmd="!sascmd";
signon task3 sascmd="!sascmd" inheritlib=(Work=locwork);
signon task4 sascmd="!sascmd" inheritlib=(Work=locwork);
signon task5 sascmd="!sascmd" inheritlib=(Work=locwork);
signon task6 sascmd="!sascmd" inheritlib=(Work=locwork);

/* run data step to input first raw file, subset the  */
/* data by quarter, and pipe it to the merge for that */
/* quarter                                            */
rsubmit task1 wait=no;
libname out1 sasesock ":pipe1";
libname out2 sasesock ":pipe2";
libname out3 sasesock ":pipe3";
libname out4 sasesock ":pipe4";
data out1.qtr1_sales out2.qtr2_sales
     out3.qtr3_sales out4.qtr4_sales;
  infile '/DATA01/mpconnect/jmk4/sales.txt' dlm=':';
  input EmpID $ 12. Qtr1 Qtr2 Qtr3 Qtr4;
  drop Qtr1 Qtr2 Qtr3 Qtr4 Qtr;
  array contrib{4} Qtr1-Qtr4;
  do Qtr=1 to 4;
    Amount_sold = Contrib{Qtr};
    if Qtr=1 then output out1.qtr1_sales;
    else if Qtr=2 then output out2.qtr2_sales;
    else if Qtr=3 then output out3.qtr3_sales;
    else if Qtr=4 then output out4.qtr4_sales;
  end;
run;
endrsubmit;

/* run data step to input second raw file, subset the  */
/* data by quarter, and pipe it to the merge for that  */
/* quarter                                             */
rsubmit task2 wait=no;
libname out1 sasesock ":pipe5";
libname out2 sasesock " :pipe6";
libname out3 sasesock " :pipe7";
libname out4 sasesock " :pipe8";
data out1.qtr1_goals out2.qtr2_goals
     out3.qtr3_goals out4.qtr4_goals;
  infile '/DATA01/mpconnect/jmk4/goals.txt' dlm=':';
  input EmpID $ 12. Qtr1 Qtr2 Qtr3 Qtr4;
  drop Qtr1 Qtr2 Qtr3 Qtr4 Qtr;
  array contrib{4} Qtr1-Qtr4;
  do Qtr=1 to 4;
    Amount_goal = Contrib{Qtr};
    if Qtr=1 then output out1.qtr1_goals;
    else if Qtr=2 then output out2.qtr2_goals;
    else if Qtr=3 then output out3.qtr3_goals;
    else if Qtr=4 then output out4.qtr4_goals;
  end;
run;
endrsubmit;

/* execute the 4 merges (one per quarter) in parallel */
/* since they are independent tasks                   */
rsubmit task3 wait=no;
libname in1 sasesock " :pipe1";
libname in2 sasesock " :pipe5";
data locwork.qtr1_comp;
  merge in1.qtr1_sales in2.qtr1_goals;
  by EmpID;
run;
endrsubmit;

rsubmit task4 wait=no;
libname in1 sasesock " :pipe2";
libname in2 sasesock " :pipe6";
data locwork.qtr2_comp;
  merge in1.qtr2_sales in2.qtr2_goals;
  by EmpID;
run;
endrsubmit;

rsubmit task5 wait=no;
libname in1 sasesock " :pipe3";
libname in2 sasesock " :pipe7";
data locwork.qtr3_comp;
  merge in1.qtr3_sales in2.qtr3_goals;
  by EmpID;
run;
endrsubmit;

rsubmit task6 wait=no;
libname in1 sasesock " :pipe4";
libname in2 sasesock " :pipe8";
data locwork.qtr4_comp;
  merge in1.qtr4_sales in2.qtr4_goals;
  by EmpID;
run;
endrsubmit;

waitfor _ALL_ task3 task4 task5 task6;

Piping Between Data Step, Sort, and Merge

This example creates two data sets with SAS data steps, sorts each of the resulting data sets, and finally merges the two sorted data sets into one final result data set. This is a perfect application to implement with multiple parallel SAS/CONNECT sessions and piping. This is because there are independent steps that can be done in parallel (creating the two data sets and sorting them) and there are also dependent steps that can be overlapped (the data step and proc sort sequences and also the two sorts with the final merge).

This application makes use of original (parent) SAS session to execute the final merge and spawns four SAS sessions which execute as follows:

signon mp1 sascmd="!sascmd";
signon mp2 sascmd="!sascmd";
signon mp1b sascmd="!sascmd";
signon mp2b sascmd="!sascmd";

/* MP1 runs a data step to create the first data set and */
/* pipe output to sort in session MP1B */
rsubmit mp1 wait=no;
libname temp '.';
libname outLib1 sasesock ":pipe1";
data outLib1.newbig;
  set temp.big;
  x="abcdefghijklm";
  y="nopqrstuvwxyz";
  z=a;
  length longvar $ 100;
  longvar="a";
  do k=1 to 200;
    longvar= trim(longvar) || "a";
  end;
  do while(int(z) >= 10);
    z=z/10;
  end;
  f=mean(c,d);
  if (f lt e/2) then do;
    g=mean(b,e);
    h= substr(x,1,z);
  end;
  else do;
    g=mean(b,d);
    h= substr(y,1,z);
  end;
  drop x y z;
run;
endrsubmit;

/* MP1B get input from data step running in MP1 and pipes */
/* output back to parent session for the merge */
rsubmit mp1b wait=no;
libname inLib1b sasesock ":pipe1";
libname outLib1b sasesock ":pipe3";
proc sort data=inLib1b.newbig out=outLib1b.bigsort
  sortsize=20K;
  by f;
run;
endrsubmit;

/* MP2 runs a data step to create the second data set and */
/* pipe output to sort in session MP2B */
rsubmit mp2 wait=no;
libname temp '.';
libname outLib2 sasesock ":pipe2";
data outLib2.newbig2;
  set temp.big2;
  x="abcdefghijklm";
  y="nopqrstuvwxyz";
  z=a;
  length longvar $ 100;
  longvar="a";
  do k=1 to 200;
    longvar=trim(longvar) || "a";
  end;
  do while(int(z) >= 10);
    z=z/10;
  end;
  f=mean(c,e);
  if (f lt e/2) then do;
    g=mean(b,e);
    h=substr(x,1,z);
  end;
  else do;
    g=mean(b,d);
    h=substr(y,1,z);
  end;
  drop x y z;
run;
endrsubmit;

/* MP2B get input from data step running in MP2 and pipes */
/* output back to parent session for the merge */
rsubmit mp2b wait=no;
libname inLib2b sasesock ":pipe2";
libname outLib2b sasesock ":pipe4";
proc sort data=inLib2b.newbig2 out=outLib2b.bigsort2
  sortsize=20K;
  by f;
run;
endrsubmit;

/* parent session performs the final merge as data comes */
/* in on the pipes from the two sorts */
libname sort1 sasesock ":pipe3";
libname sort2 sasesock ":pipe4";
data merged;
  merge sort1.bigsort sort2.bigsort2;
  by f;
run;

signoff mp1;
signoff mp2;
signoff mp1b;
signoff mp2b;