Background Information

Bulk loading is an automated approach to adding and maintaining user identities that complements the manual steps that you can perform within SAS^® Management Console. Specifically, bulk loading refers to the importation and synchronization of users and groups into the SAS^® Metadata Repository from external sources such as the following:

• the Lightweight Directory Access Protocol (LDAP)
• a host password file
• another data source that contains user and group information

When a filter for a large group is defined prior to the call for the %LDAPEXTRGROUPS macro, the following message might be reported in the log:

Because the group has more than 1,500 members, it exceeds the limit that the program can extract. Therefore, no memberships are extracted into the table and 0 observations are returned.

You might also see the following error written into the importad.sas program when there are no observations returned in the work.ldapgrps data set:

Note that if multiple group filters are defined and some groups have less than 1,500 members, the error might not be produced.

When you search for a group that has more than 1,500 members, Active Directory returns two member attributes, one of which is a member range, as shown in this example:

...more lines...

As you can see, 0 NUM values are returned for the member attribute.

You encounter this issue because the SAS sample code (importad.sas) uses a DO loop within the %LDAPEXTRGROUPS macro to process members of a group:

Within the program execution phase, numValues is assigned a value of 0. The DO loop is essentially do i = 1 to 0, which is a loop that never executes. Because the loop does not execute, the output &extractlibref..ldapgrps; statement never executes, and the group name is never written to the IDGRPS table. This behavior explains why this group is not added to the metadata.

Sample Information

The solution is to process large groups using Range Retrieval.

In the SAS code, this means using the attribute member;range=0-1499 (where 0-1,499 is a range of entries) rather than simply member.

Code Changes

There are several places in the code where you need to make this change.

1. In the attribute list provided in the %LDAPEXTRGROUPS macro, use the member range as the attribute name:

       attrs="name description groupType distinguishedName " ||

             "sAMAccountName member;range=0-1499 whenChanged whenCreated" ||

             "displayName";

2. In the conditional assignment of memberindex, use the member range as the attribute name:

                  /********************************************************************/

             /* If the attrName is member, then remember the aIndex so that you  */

             /* can loop through all the members after the group attributes are  */

             /* retrieved.                                                       */

             /********************************************************************/

             if (attrName = 'member;range=0-1499') then

                memberindex = aIndex;

             else do;  /* Gets the first value of the attribute. */

                call ldaps_attrValue(shandle, eIndex, aIndex, 1, value, rc);

                if rc NE 0 then do;

                   msg = sysmsg();

                   put msg;

                end;

             end;

3. In the conditional assignment of member=value, use the member range as the attribute name:

             /* Extracts the member - Member for Group. */

             if (attrName = 'member;range=0-1499')  then do;

             /* Extracts all the members of the group. */

              member=value; /* DN of the group members. */

             end;

Method

You can use the following method to implement this change:

1. Make a copy of the existing %LDAPEXTRGROUPS macro.
2. Use a keyword parameter to provide the range value of the member attribute.
3. For groups that you know are large or have more than 1,500 members, call the modified "large group" macro on the Full Code tab rather than calling the original %LDAPEXTRGROUPS macro.

Full-Code Tab Instructions

The sample provided on the Full Code tab is a copy of the %LDAPEXTRGROUPS macro with the member range macro variables in place, as described above. All other code in the macro definition is the same as in the original %LDAPEXTRGROUPS macro.

To use the new macro, complete the following steps:

1. Insert the macro definition (the entire content of the attached program) after the %MEND statement for the %LDAPEXTRGROUPS macro, and before the data &extractlibref..ldapgrps statement of the next step in your importad.sas program.
2. Call the macro after the filter for the large group and provide the range of members to fetch. You call the macro more than once for the same filter.
3. Repeat the filter and call as many times as you need to extract all members. Note that the upper-bound of the last range can include a wildcard (*). For example, you can use the following code:

Note also that the range that you specify does not have to be 1,500 members. For example, you can use range values of range=1-999, range=1000-1999, range=2000-2999, range=3000-3999, range=4000-*, as long as no range exceeds 1,500 members.

For more information, see "Range Retrieval Results" at Searching Using Range Retrieval for an explanation of why these results are returned for large groups.

---

These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.

%macro ldapextr_Large_groups(range=0-1499);

       shandle=0;
       num=0;

       attrs="name description groupType distinguishedName " ||
             "sAMAccountName whenChanged whenCreated" ||
             "displayName member;range=&range" ;

       /*****************************************************************/
       /* Calls the SAS interface to search the LDAP directory. Upon    */
       /* successful return, the shandle variable will contain a search */
       /* handle that identifies the list of entries returned in the    */
       /* search. The num variable will contain the total number of     */
       /* result entries found during the search.                       */
       /*****************************************************************/
       call ldaps_search(handle,shandle,filter, attrs, num, rc);
       if rc NE 0 then do;
          msg = sysmsg();
          put msg;
          put filter=;
       end;

       do eIndex = 1 to num;

          numAttrs=0;
          entryname='';

          call ldaps_entry(shandle, eIndex, entryname, numAttrs, rc);
          if rc NE 0 then do;
             msg = sysmsg();
             put msg;
          end;

          /* Initializes the entry variables. */
          name="";
          description="";
          groupType="";
          distinguishedName="";
          sAMAccountName="";
          member=""; /* DN of the group members. */
          whenChanged="";
          whenCreated="";
          displayname="";


          /***********************************************************************/
          /* For each attribute, retrieves name and values.                       */
          /* Initializes the member attribute index to 0. It will get set in the  */
          /* loop below and then be used to retrieve group members after the     */
          /* group attributes are set.                                           */
          /***********************************************************************/
          memberindex = 0;

          if (numAttrs > 0) then do aIndex = 1 to numAttrs;

             attrName='';
             numValues=0;

             call ldaps_attrName(shandle, eIndex, aIndex, attrName, numValues, rc);
             if rc NE 0 then do;
                put aIndex=;
                msg = sysmsg();
                put msg;
             end;

             /********************************************************************/
             /* If the attrName is a member range, then remember the attribute   */
	     /* index so that we can loop through all the members after the      */
             /* group attributes are retrieved.                                  */
             /********************************************************************/
             if (attrName = "member;range=&range") then
                memberindex = aIndex;

             else do;  /* Gets the first value of the attribute. */
                call ldaps_attrValue(shandle, eIndex, aIndex, 1, value, rc);
                if rc NE 0 then do;
                   msg = sysmsg();
                   put msg;
                end;
             end;

             /* Extracts the description - Description. */
             if (attrName = 'description')  then
                description=value;
             /* Extracts the name - RDN  (relative distinguished name).  */
             if (attrName = 'name')  then
                name=value;
             /* Extracts the groupType - Group-Type.   */
             if (attrName = 'groupType')  then
                groupType=value;
             /* Extracts the distinguishedName - Obj-Dist-Name. */
             if (attrName = 'distinguishedName')  then
                 distinguishedName=value;
             /* Extracts the sAMAccountName - SAM-Account-Name. */
             if (attrName = 'sAMAccountName')  then
                sAMAccountName=value;

             /* Extracts the member - Member for Large Group. */
             if (attrName = "member;range=&range")  then do;
                /* Extracts all the members of the group. */
                member=value; /* DN of the group members. */
             end;

             /* Extracts the whenChanged - When-Changed. */
             if (attrName = 'whenChanged')  then
                whenChanged=value;
             /* Extracts the whenCreated - When-Created. */
             if (attrName = 'whenCreated')  then
                whenCreated=value;

             /* Extracts the displayname - displayName. */
             if (attrName = 'displayName')  then
                displayname=value;

          end;  /* End of attributes loop. */

          /* ... Group defined with no members. */
          if memberindex = 0 then do;
             member="";
             output &extractlibref..ldapgrps;  /* Writes out the Group Name Entry. */
          end;

          /* ... When the Group has members, then retrieves each one. */
          else do;

             attrName='';
             numValues=0;

             call ldaps_attrName(shandle, eIndex, memberindex, attrName, numValues, rc);
             if rc NE 0 then do;
                put aIndex=;
                msg = sysmsg();
                put msg;
             end;

             do i = 1 to numValues;         /* Gets all the members of this group. */

                call ldaps_attrValue(shandle, eIndex, memberindex, i, value, rc);
                if rc NE 0 then do;
                   msg = sysmsg();
                   put msg;
                end;
                member = value;
                output &extractlibref..ldapgrps;  /* Writes out the Group Member Entry. */
             end;

          end;  /* End of members loop. */

       end;  /* End of entry loop. */

       /* Free search resources. */
       if shandle NE 0 then do;
          call ldaps_free(shandle,rc);
          if rc NE 0 then do;
             msg = sysmsg();
             put msg;
          end;
       end;
%mend;

---

These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.

Operating System and Release Information