The HPDS2 Procedure

Example 7.2 Aggregate Result Data Set

This example illustrates how the intermediate result data that are generated from the DS2 code running in parallel on separate grid nodes can be aggregated into a final result data set. In this case, the aggregation is done by a second-stage PROC DS2 program that executes on the SAS client.

This example uses a DATA step program that runs on the SAS client to generate a sample data set that consists of dimensional information for each of 200 objects (closed cylinders). These data are used by the HPDS2 procedure to calculate the volume and surface area of each object. The second-stage DS2 procedure aggregates these results, summing the total volume and surface area for all objects and computing the average volume and surface area. In this example, the DS2 code running in parallel on the grid is used to perform the row-independent and computationally intensive portion of the processing, whereas the work done by the second-stage DS2 procedure is limited to the final result aggregation and summary.

libname applianc &ENGINE
        server = "&GRIDDATASERVER"
        user   = &USER 
        password = &PASSWORD 
        database = &DATABASE;

data obj_dims;
   do id=1 to 200;
      radius = ranuni(1) * 10;
      height = ranuni(1) * 20;
      output;
   end;
run;

%let pi=3.14159;
proc hpds2 data=obj_dims
           out=applianc.obj_comps;
   performance host="&GRIDHOST" install="&GRIDINSTALLLOC";
   data DS2GTF.out;
      method run();
         set DS2GTF.in;
         volume = &pi * radius**2 * height;
         area = (2 * &pi * radius**2) + (2 * &pi * radius * height);
      end;
   enddata;
run;
proc print data=applianc.obj_comps (obs=20);
   title1 'Volumes and Areas';
run;
data obj_comps;
   set applianc.obj_comps;
run;
proc ds2;
   data obj_totals (keep = (ncount vsum asum vmean amean));
      dcl double ncount vsum asum vmean amean;
      method init();
         ncount = 0;
         vsum = 0;
         asum = 0;
      end;
      method run();
         set {select volume, area from obj_comps};
         ncount + 1;
         vsum + volume;
         asum + area;
      end;
      method term();
         if ncount ne 0 then do;
            vmean = vsum/ncount;
            amean = asum/ncount;
         end;
         output;
      end;
   enddata;
run;
quit;
proc print data=obj_totals;
   title1 'Total Volume and Area';
run;

Output 7.2.1 shows a subset of the volumes and areas that are computed by the HPDS2 procedure.

Output 7.2.1: Computed Volumes and Areas

Volumes and Areas

Obs id radius height volume area
1 1 1.84963 19.4018 208.53 246.97
2 2 3.99824 5.1880 260.55 230.77
3 3 9.21603 19.3855 5172.67 1656.20
4 4 5.42979 10.6338 984.93 548.03
5 5 0.49794 1.3313 1.04 5.72
6 6 8.19319 10.4774 2209.58 961.15
7 7 8.53394 1.3437 307.43 529.64
8 8 9.57024 5.9439 1710.27 932.89
9 9 2.72612 13.7986 322.16 283.05
10 10 9.76765 4.5302 1357.82 877.48
11 11 6.88237 8.2553 1228.45 654.60
12 12 5.58554 5.7445 563.03 397.63
13 13 4.75789 16.8997 1201.87 647.45
14 14 6.34524 11.8073 1493.47 723.71
15 15 5.82582 7.5403 803.99 489.26
16 16 7.28362 10.1321 1688.66 797.02
17 17 9.31214 18.5824 5062.32 1632.10
18 18 5.89660 5.9445 649.33 438.70
19 19 3.91042 9.4486 453.90 328.23
20 20 6.79526 3.3618 487.67 433.66


Output 7.2.2 shows the aggregated results that are produced by the second-stage DS2 program.

Output 7.2.2: Computed Total Volume and Area

Total Volume and Area

Obs NCOUNT VSUM ASUM VMEAN AMEAN
1 200 209883.99 104680.26 1049.42 523.401