You are currently reviewing an older revision of this page.

KB-XXXX High Availability Service Manager cluster refuses to shutdown

Symptoms

While attempting to shutdown the engines in an HA cluster, the service manager process refuses to carry out the shutdown and ignores the stop script.

Users may see the following message in the service manager log:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked

Additionally, Users may see that every "'shutdown-cluster' invoked" message is immediately followed by a "Cluster shutdown already in progress!" message. For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

However, the shutdown does not proceed as expected. 

Summary of symptoms:

  • Failed graceful shutdown for HA sites
  • Service Manager is unresponsive to the stop script, even with the -c option
  • Engines fail to promote to PRIMARY on a particular node (even if there are no other primaries and the engine is a RUNNING REPLICA)

Cause

These symptoms indicate that service manager was in a "cluster-stop" state prior to initiating the graceful shutdown. Typically, a user is trying to shutdown service manager on just one node in an HA configuration, but accidentally does a cluster stop which affects all other service managers in a cluster. Service manager could also get into this state if there was an issue with an intentional cluster shutdown, but only the engines are restarted and not service manager.

Action

This issue can be remediated by restarting service manager by running the following:

<APPIAN_HOME>/services/stop.sh -p XXXXXXX -s manager

<APPIAN_HOME>/services/start.sh -p XXXXXXX -s all

Affected Versions

This article applies to all versions of Appian running with a High Availability Topology

Last Reviewed: May 2020