On some Microsoft Windows Server 2016 and 2019 systems, the SAS^® Environment Manager Agent might be responsible for exhausting all available ephemeral ports. The number of ports is limited to approximately 64,000. The port exhaustion has been linked to a SAS Environment Manager Agent process that leaves connections open to SAS^® Metadata Server, after its metric collection thread times out. When the limit is reached, which typically occurs in less than 24 hours, the operating system becomes unresponsive to network traffic, including remote desktop connections. A reboot of the server is then needed to restore responsiveness to network traffic.

To confirm that the issue is present, review the SAS Environment Manager Agent agent.log file for particular error messages. The file is located in this default location:

In the file, you will see error messages similar to the following that are logged many times:

Also, you can see that the below netstat command will report back a count that is typically much greater than 1,000, and the count steadily increases over time. The process ID is returned as well. Note: The vast majority of the connections are from the same process ID.

Running the netstat command as shown below returns the connection list and the associated process ID:

In the example above, process ID 5468 is linked to the port exhaustion. The following PowerShell command reveals that this process ID is sas.exe and is specifically from the SAS Metadata Server:

Also, TCPIP Event ID 4227 and 4231 might be logged in the System Event Log in Windows Event Viewer, indicating that the ephemeral port-count limit was reached.

Stopping the SAS Environment Manager Agent service temporarily mitigates the problem. However, the problem manifests again shortly after the SAS Environment Manager Agent service is restarted.

To work around this problem, take these steps to increase the schedule thread time-out value in the agent.properties file for the SAS Environment Manager Agent:

1. Stop the SAS Environment Manager Agent service by using the Windows Services control panel (services.msc). The agent can also be stopped by accessing the command prompt as an administrator and then running this command: 
   
      SAS-configuration-directory\Lev1\Web\SASEnvironmentManager\agent-5.8.0-EE\bin\hq-agent.bat stop
    
2. Find the agent.properties file on the server. The file is in this location:
   
      SAS-configuration-directory\Lev1\Web\SASEnvironmentManager\agent-5.8.0-EE\conf
    
3. Edit the file and add these lines at the bottom of the file:
   
      scheduleThread.poolsize.vsphere=2​​​​​​​
      scheduleThread.cancelTimeout=60000
    
4. Start the SAS Environment Manager Agent by using the Windows Services control panel (services.msc). The agent can also be started by accessing the command prompt as an administrator and then running this command: 
   
      SAS-configuration-directory\Lev1\Web\SASEnvironmentManager\agent-5.8.0-EE\bin\hq-agent.bat start

Operating System and Release Information