SUPPORT / SAMPLES & SAS NOTES
Problem Note 66706: Elasticsearch has multiple master nodes or the elasticsearch_check_cluster deployment step fails during an installation
 |  |  |  |  |
In a multi-node Elasticsearch deployment of SAS® Visual Investigator, the elasticsearch_check_cluster step might fail. When you look at your Elasticsearch nodes, you notice that there are multiple masters with nodes. All nodes should have a single master.
The following commands demonstrate an example of what the problem looks like by querying the Elasticsearch nodes directly:
$ sudo curl https://elasticsearch-node-1:port/_cat/nodes?v --cacert /opt/sas/viya/config/etc/SASSecurityCertificateFramework/cacerts/trustedcerts.pem --key /opt/sas/viya/config/etc/elasticsearch/default/keys/searchguard//sghealthcheck-key.pem --cert /opt/sas/viya/config/etc/elasticsearch/default/certs/searchguard//sghealthcheck-cert.pem
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.20.49 10 81 5 0.95 1.26 3.83 mdi * Y4peTZd
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.20.49 10 81 5 0.95 1.26 3.83 mdi * Y4peTZd
$ sudo curl https://elasticsearch-node-2:port/_cat/nodes?v --cacert /opt/sas/viya/config/etc/SASSecurityCertificateFramework/cacerts/trustedcerts.pem --key /opt/sas/viya/config/etc/elasticsearch/default/keys/searchguard//sghealthcheck-key.pem --cert /opt/sas/viya/config/etc/elasticsearch/default/certs/searchguard//sghealthcheck-cert.pem
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.25.186 12 33 1 0.04 0.15 0.20 mdi * o2hp4Cn
15.124.26.3 12 33 0 0.65 0.41 0.34 mdi - OHQFYmm
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.25.186 12 33 1 0.04 0.15 0.20 mdi * o2hp4Cn
15.124.26.3 12 33 0 0.65 0.41 0.34 mdi - OHQFYmm
$ sudo curl https://elasticsearch-node-3:port/_cat/nodes?v --cacert /opt/sas/viya/config/etc/SASSecurityCertificateFramework/cacerts/trustedcerts.pem --key /opt/sas/viya/config/etc/elasticsearch/default/keys/searchguard//sghealthcheck-key.pem --cert /opt/sas/viya/config/etc/elasticsearch/default/certs/searchguard//sghealthcheck-cert.pem
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.26.3 12 33 0 0.68 0.42 0.35 mdi - OHQFYmm
15.124.25.186 12 33 0 0.04 0.14 0.20 mdi * o2hp4Cn
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.26.3 12 33 0 0.68 0.42 0.35 mdi - OHQFYmm
15.124.25.186 12 33 0 0.04 0.14 0.20 mdi * o2hp4Cn
When this issue happens, restart the Elasticsearch service on your master node. When you query the Elasticsearch master node, you should see a result similar to the following:
$ sudo curl https://elasticsearch-master-node:port/_cat/nodes?v --cacert /opt/sas/viya/config/etc/SASSecurityCertificateFramework/cacerts/trustedcerts.pem --key /opt/sas/viya/config/etc/elasticsearch/default/keys/searchguard//sghealthcheck-key.pem --cert /opt/sas/viya/config/etc/elasticsearch/default/certs/searchguard//sghealthcheck-cert.pem
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.26.3 12 33 1 0.09 0.28 0.32 mdi - OHQFYmm
15.124.25.186 9 33 1 0.01 0.06 0.15 mdi * o2hp4Cn
15.124.20.49 9 81 9 0.90 1.38 2.97 mdi - Y4peTZd
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
15.124.26.3 12 33 1 0.09 0.28 0.32 mdi - OHQFYmm
15.124.25.186 9 33 1 0.01 0.06 0.15 mdi * o2hp4Cn
15.124.20.49 9 81 9 0.90 1.38 2.97 mdi - Y4peTZd
Then rerun your deployment.
Click the Hot Fix tab in this note for a link to instructions about accessing and applying the software update.
Operating System and Release Information
| Product Family | Product | System | Product Release | SAS Release | ||
| Reported | Fixed* | Reported | Fixed* | |||
| SAS System | SAS Visual Investigator | Linux for x64 | 10.7 | Viya | ||