KB-2113 High Availability Service Manager cluster fails to shutdown

Symptoms

While attempting to shutdown the engines in a High Availability cluster, the service manager process fails to respond to requests for shutdown via the stop.bat (.sh) script located in <APPIAN_HOME>/services/bin/. This is also true when passing the -c option to the stop script. Engines on a particular node will also fail to promote from REPLICA to PRIMARY.

After the shutdown attempt, every "'shutdown-cluster' invoked" message may be immediately followed by a "Cluster shutdown already in progress!" message in the service_manager.log (located in <APPIAN_HOME>/logs/). For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

Cause

These symptoms indicate that service manager was in a "cluster-stopping" state prior to initiating the graceful shutdown. This can occur when a service manager cluster stop is initiated via the stop script, or if there was an issue with a previous cluster shutdown, and only the engines were restarted.

Action

This issue can be remediated by restarting service manager:

  1. <APPIAN_HOME>/services/stop.sh -p <password> -s manager
  2. <APPIAN_HOME>/services/start.sh -p <password> -s all

If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

Affected Versions

This article applies to Appian 18.1 and higher running in a High Availability Topology.

Last Reviewed: May 2020

Related
Recommended