SUPPORT / SAMPLES & SAS NOTES
 

Support

Problem Note 67692: SAS® Workload Orchestrator might unexpectedly stop running when you use Kerberos authentication

DetailsHotfixAboutRate It

When you use the SAS Workload Orchestrator grid provider and Kerberos integration with OneIdentity/Quest/VAS Safeguard Authentication Services, the SAS Workload Orchestrator daemon (sgmg) might stop unexpectedly.

In the Grid console log (SAS-Configuration-Directory/Levn/Grid/Logs/grid.console.hostname.log), messages similar to the following might be displayed:

*** Error in `SAS-Installation-Directory/SASFoundation/9.4/utilities/bin/sgmg': munmap_chunk(): invalid pointer: 0x00002b46f9338c10 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7f804)[0x2b46f8b90804]
/opt/quest/lib64/libvas.so.4(vassym_krb5_data_free+0xd)[0x2b471636e72d]
/opt/quest/lib64/libvas.so.4(vassym_krb5_free_address+0x53)[0x2b471636ed03]
/opt/quest/lib64/libvas.so.4(vassym_krb5_free_addresses+0x38)[0x2b471636ed88]
/opt/quest/lib64/libvas.so.4(vassym_krb5_free_cred_contents+0x62)[0x2b4716372572]
<SAS-Installation-Directory>/SASFoundation/9.4/sasexe/tksecgss.so(+0xdb68)[0x2b4715db3b68]
<SAS-Installation-Directory>/SASFoundation/9.4/sasexe/tksecgss.so(+0xedc1)[0x2b4715db4dc1]
<SAS-Installation-Directory>/SASFoundation/9.4/utilities/bin/sgmg(+0xa287c)[0x563ad2aff87c]
<SAS-Installation-Directory>/SASFoundation/9.4/utilities/bin/sgmg(+0x9c9b4)[0x563ad2af99b4]
<SAS-Installation-Directory>/SASFoundation/9.4/utilities/bin/sgmg(+0x993f0)[0x563ad2af63f0]
<SAS-Installation-Directory>/SASFoundation/9.4/utilities/bin/sgmg(utilThreadHandler+0x726)[0x563ad2b1fdb6]
<SAS-Installation-Directory>/SASFoundation/9.4/sasexe/tkmk.so(sktMain+0x94)[0x2b46f930d514]
<SAS-Installation-Directory>/SASFoundation/9.4/sasexe/tkmk.so(bktMain+0x71)[0x2b46f930f2b1]
/lib64/libpthread.so.0(+0x7ea5)[0x2b46f81bfea5]
/lib64/libc.so.6(clone+0x6d)[0x2b46f8c0f8cd]

The Grid log (SAS-Configuration-Directory/Levn/Grid/Logs/Grid_date_hostname_pid.log) might contain corresponding messages similar to the following:

WARN  [00000014] Audit.Authentication (tksecgss.c:4701) - Kerberos failure in function krb5_get_credentials: encryption type ######### not supported (96C73A96).
ERROR [00000014] Audit.Authentication (tksecgss.c:4809) - Failed to get kerberos credentials for cache /tmp/krb5cc_XXXXXX.

This issue is caused by differences between the MIT Kerberos and QAS/VAS libraries when you attempt to renew a ticket.

To work around the issue, instruct SAS to use the MIT Kerberos libraries instead by making the following changes:

In the Metadata and Compute Tier configuration directories, add this line to the SAS-Configuration-Directory/Levn/level_env_usermods.sh file:

export TKSECURE_GSSAPI_LIBRARY=/lib64/libgssapi_krb5.so.2

In the Metadata and Compute Tier installation directories, add this line to the SAS-Installation-Directory/SASFoundation/9.4/utilities/bin/sasauth.conf file:

gssLibrary=/lib64/libgssapi_krb5.so.2

In the Compute Tier configuration directory, add this line to the SAS-Configuration-Directory/Levn/Grid/sgmg_usermods.sh file:

export TKSECURE_GSSAPI_LIBRARY=/lib64/libgssapi_krb5.so.2

After you make the adjustments, restart the following associated SAS services:

  • Metadata Server
  • Object Spawner
  • SGMG/SAS Workload Orchestrator
  • SAS OLAP Server
  • Connect Spawner
  • Launcher Server


Operating System and Release Information

Product FamilyProductSystemProduct ReleaseSAS Release
ReportedFixed*ReportedFixed*
SAS SystemSAS Workload Orchestrator64-bit Enabled Solaris9.49.4 TS1M6
Linux for x649.49.4 TS1M6
Solaris for x649.49.4 TS1M6
* For software releases that are not yet generally available, the Fixed Release is the software release in which the problem is planned to be fixed.