Overview

Under the following conditions, you must make changes in the LASRMonitor configuration in order to support the Explore HDFS tab in SAS Visual Analytics Administrator:

• You applied the SAS Visual Analytics container Hot Fix B2B010.
• You applied the SAS Plug-ins for Hadoop Hot Fix D9U001 (or the SAS Plug-ins for Hadoop package that is included with a SAS^® 9.4M6 (TS1M6) software depot).

The Explore HDFS tab is deployed only in the following SAS Visual Analytics environments:

• You selected Hadoop (co-located HDFS) as the SAS Visual Analytics Data Provider during SAS Visual Analytics deployment.
• You use the distributed SAS^® LASR Analytic Server. 

The LASRMonitor process has always required passwordless Secure Shell (SSH) capabilities for the user that is running the process in order to connect to the High-Performance Analytics Environment (TKGrid). These capabilities are also required to invoke other processes that return information for resource and memory monitors that are displayed in SAS Visual Analytics Administrator. The significant change is that SSH is now also used to connect to Hadoop when you use the HDFS browser.  

Quick Setup

The LASRMonitor component is deployed on the SAS Visual Analytics compute tier. Here are the instructions to make the required changes.

1. In many cases, the only change necessary is to add the following property to Compute-Tier-Config-Dir/LevX/Applications/SASVisualAnalytics/HighPerformanceConfiguration/service.properties:

Cloudera Example:

In Cloudera environments, add the following property:

Hortonworks Example:

In Hortonworks environments, add the following property:

Note: The path might vary in your Hadoop cluster. Ensure that the path contains bin/hadoop. The slash (/) at the end of the path is required.

2. Run the following command to restart LASRMonitor and make the change effective:

3. You might need to press Enter to return to a command prompt after starting or restarting LASRMonitor.

HDFS-Related Properties List for Service.properties

Here are some additional properties that might be useful in certain scenarios:

Property Testing 

This section describes how to test your properties.

1. In order to test your service.properties options, construct an SSH command using your selected options in this format: 

Here is an example.

The example service.properties contains the following:

Here is the example command using the specified values and the default values for those that are not specified:

2. Run the SSH command on the LASRMonitor system as the user that runs the LASRMonitor process. The command should successfully execute a hadoop fs -ls / command on the Hadoop NameNode and produce a file that lists output.

Configuration for Hadoop Clusters That Use Kerberos Authentication

In order for SSH sessions initiated by LASRMonitor to have valid, unexpired Kerberos credentials anytime that the HDFS Browser is used, a keytab is required for either of the following user scenarios:

• a user that is running LASRMonitor
• a user that is specified by additional SSH options in the hadoop.plugins.ssh.options property

Add a kinit command in the user's ~/.ssh/rc file on the Hadoop NameNode so that the keytab is used to obtain a Kerberos TGT whenever SSH is invoked.

Here is an example:

1. LASRMonitor is started by the lasrmon user on the SAS Visual Analytics compute tier.
2. The Kerberos administrator creates a keytab file for the lasrmon user and places the keytab at /home/lasrmon/lasrmon.keytab on the Hadoop NameNode.
3. The lasrmon user or administrator creates /home/lasrmon/.ssh/rc on the Hadoop NameNode with the following contents:

4. When the HDFS Browser is accessed, the LASRMonitor that is running under the lasrmon identity sends an SSH command to the Hadoop NameNode, invoking the kinit command in /home/lasrmon/.ssh/rc. This action creates a new Kerberos ticket that ensures that the lasrmon user can authenticate to Hadoop and retrieve information that is displayed in the HDFS browser.

Problem Symptoms

If you have not applied SAS Visual Analytics container Hot Fix B2B010, but you have updated the SAS Plug-ins for Hadoop, the Explore HDFS tab might display no content and no error.

Under any of the following conditions, clicking the Explore HDFS tab returns "An error occurred when the Hadoop content explorer tried to contact the SAS LASR Analytic Server Monitor. Please contact your system administrator."

• You have applied SAS Visual Analytics container Hot Fix B2B010, but you have not updated the SAS Plug-ins for Hadoop.
• You have updated both SAS Visual Analytics container Hot Fix B2B010 and SAS Plug-ins for Hadoop but have not added the properties described in this SAS Note to the service.properties file.
• You have updated both SAS Visual Analytics container Hot Fix B2B010 and SAS Plug-ins for Hadoop and have added the properties described in this SAS Note to the service.properties file. However, the properties cannot be used to make a successful connection to Hadoop and obtain a file listing.

Here is a screen shot of the error that you see:

Troubleshooting

Here are some troubleshooting steps that you can take when you see errors:

1. Test your service.properties options as described in the Property Testing section above.
2. Set hadoop.plugins.ssh.stderr.redirection in service.properties to a path on the Hadoop NameNode to which the LASRMonitor user has Write access.

Here is an example:

3. Use the following command to restart LASRMonitor:

4. Reproduce the problem, and then review the following logs or send them to SAS Technical Support:

From the Hadoop NameNode:

From the SAS Visual Analytics compute tier:

Operating System and Release Information

A fix for this issue for SAS Visual Analytics 7.4 is available at: