KB-2113 High Availability Service Manager cluster fails to shutdown
Symptoms
While attempting to shutdown the engines in a High Availability cluster, the service manager process fails to respond to requests for shutdown via the stop.bat (.sh) script located in <APPIAN_HOME>/services/bin/. This is also true when passing the -c option to the stop script. Engines on a particular node will also fail to promote from REPLICA to PRIMARY.
After the shutdown attempt, every "'shutdown-cluster' invoked" message may be immediately followed by a "Cluster shutdown already in progress!" message in the service_manager.log (located in <APPIAN_HOME>/logs/). For example:
INFO com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
Cause
These symptoms indicate that service manager was in a "cluster-stopping" state prior to initiating the graceful shutdown. This can occur when a service manager cluster stop is initiated via the stop script, or if there was an issue with a previous cluster shutdown, and only the engines were restarted.
Action
This issue can be remediated by restarting service manager:
<APPIAN_HOME>/services/stop.sh -p <password> -s manager<APPIAN_HOME>/services/start.sh -p <password> -s all
If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.
Affected Versions
This article applies to Appian 18.1 and higher running in a High Availability Topology.
Last Reviewed: May 2020