While attempting to shutdown the engines in a High Availability cluster, the service manager process fails to respond to requests for shutdown via the stop.bat (.sh) script located in <APPIAN_HOME>/services/bin/. This is also true when passing the -c option to the stop script. Engines on a particular node will also fail to promote from REPLICA to PRIMARY.
After the shutdown attempt, every "'shutdown-cluster' invoked" message may be immediately followed by a "Cluster shutdown already in progress!" message in the service_manager.log (located in <APPIAN_HOME>/logs/). For example:
INFO com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
These symptoms indicate that service manager was in a "cluster-stopping" state prior to initiating the graceful shutdown. This can occur when a service manager cluster stop is initiated via the stop script, or if there was an issue with a previous cluster shutdown, and only the engines were restarted.
This issue can be remediated by restarting service manager:
<APPIAN_HOME>/services/stop.sh -p <password> -s manager
<APPIAN_HOME>/services/start.sh -p <password> -s all
If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.
This article applies to Appian 18.1 and higher running in a High Availability Topology.
Last Reviewed: May 2020
© 2020 Appian. All rights reserved.