KB-2113 High Availability Service Manager cluster fails to shutdown

James Lee — Tue, 12 May 2020 10:58:09 GMT

Current Revision posted to Appian Knowledge Base by James Lee on 5/12/2020 10:58:09 AM

Symptoms

While attempting to shutdown the engines in a High Availability cluster, the service manager process fails to respond to requests for shutdown via the stop.bat (.sh) script located in <APPIAN_HOME>/services/bin/. This is also true when passing the -c option to the stop script. Engines on a particular node will also fail to promote from REPLICA to PRIMARY.

After the shutdown attempt, every "'shutdown-cluster' invoked" message may be immediately followed by a "Cluster shutdown already in progress!" message in the service_manager.log (located in <APPIAN_HOME>/logs/). For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

Cause

These symptoms indicate that service manager was in a "cluster-stopping" state prior to initiating the graceful shutdown. This can occur when a service manager cluster stop is initiated via the stop script, or if there was an issue with a previous cluster shutdown, and only the engines were restarted.

Action

This issue can be remediated by restarting service manager:

<APPIAN_HOME>/services/stop.sh -p <password> -s manager
<APPIAN_HOME>/services/start.sh -p <password> -s all

If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

Affected Versions

This article applies to Appian 18.1 and higher running in a High Availability Topology.

Last Reviewed: May 2020

Tags: engines, infrastructure, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

James Lee — Tue, 12 May 2020 10:56:26 GMT

Revision 9 posted to Appian Knowledge Base by James Lee on 5/12/2020 10:56:26 AM

Symptoms

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

Cause

Action

This issue can be remediated by restarting service manager:

<APPIAN_HOME>/services/stop.sh -p <password> -s manager
<APPIAN_HOME>/services/start.sh -p <password> -s all

If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

Affected Versions

This article applies to Appian 18.1 and higher running in a High Availability Topology.

Last Reviewed: May 2020

Tags: engines, infrastructure, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

Sean Kim — Mon, 11 May 2020 07:59:32 GMT

Revision 8 posted to Appian Knowledge Base by Sean Kim on 5/11/2020 7:59:32 AM

Symptoms

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

Cause

Action

This issue can be remediated by restarting service manager:

<APPIAN_HOME>/services/stop.sh -p <password> -s manager
<APPIAN_HOME>/services/start.sh -p <password> -s all

If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

Affected Versions

This article applies to Appian 18.1 and higher running in a High Availability Topology.

Last Reviewed: May 2020

Tags: engines, infrastructure, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

Philip Moss — Thu, 07 May 2020 13:37:48 GMT

Revision 7 posted to Appian Knowledge Base by Philip Moss on 5/7/2020 1:37:48 PM

Symptoms

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

Cause

Action

This issue can be remediated by restarting service manager:

<APPIAN_HOME>/services/stop.sh -p <password> -s manager
<APPIAN_HOME>/services/start.sh -p <password> -s all

If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

Affected Versions

This article applies to all versions of Appian running Service Manager with a High Availability Topology.

Last Reviewed: May 2020

Tags: engines, infrastructure, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

James Lee — Wed, 06 May 2020 13:52:27 GMT

Revision 6 posted to Appian Knowledge Base by James Lee on 5/6/2020 1:52:27 PM

Symptoms

The following message may print in the service_manager.log located in <APPIAN_HOME>/logs/:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked

Additionally, every "'shutdown-cluster' invoked" message may be immediately followed by a "Cluster shutdown already in progress!" message. For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

Cause

Action

This issue can be remediated by restarting service manager:

<APPIAN_HOME>/services/stop.sh -p <password> -s manager
<APPIAN_HOME>/services/start.sh -p <password> -s all

If issues still occur, open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

Affected Versions

This article applies to all versions of Appian running Service Manager with a High Availability Topology.

Last Reviewed: May 2020

Tags: engines, infrastructure, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

Sean Kim — Mon, 04 May 2020 14:09:45 GMT

Revision 4 posted to Appian Knowledge Base by Sean Kim on 5/4/2020 2:09:45 PM

Symptoms

While attempting to shutdown the engines in an HA cluster, the service manager process refuses to carry out the shutdown and ignores the stop script.

Users may see the following message in the service manager log:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked

Additionally, Users may see that every "'shutdown-cluster' invoked" message is immediately followed by a "Cluster shutdown already in progress!" message. For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

However, the shutdown does not proceed as expected.

Summary of symptoms:

Failed graceful shutdown for HA sites
Service Manager is unresponsive to the stop script, even with the -c option
Engines fail to promote to PRIMARY on a particular node (even if there are no other primaries and the engine is a RUNNING REPLICA)

Cause

These symptoms indicate that service manager was in a "cluster-stop" state prior to initiating the graceful shutdown. Typically, a user is trying to shutdown service manager on just one node in an HA configuration, but accidentally does a cluster stop which affects all other service managers in a cluster. Service manager could also get into this state if there was an issue with an intentional cluster shutdown, but only the engines are restarted and not service manager.

Action

This issue can be remediated by restarting service manager by running the following:

<APPIAN_HOME>/services/stop.sh -p XXXXXXX -s manager

<APPIAN_HOME>/services/start.sh -p XXXXXXX -s all

If issues still occur, please contact Appian Support by creating a support case or emailing support@appian.com.

Affected Versions

This article applies to all versions of Appian running with a High Availability Topology

Last Reviewed: May 2020

Tags: engines, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

Sean Kim — Mon, 04 May 2020 14:07:49 GMT

Revision 3 posted to Appian Knowledge Base by Sean Kim on 5/4/2020 2:07:49 PM

Symptoms

While attempting to shutdown the engines in an HA cluster, the service manager process refuses to carry out the shutdown and ignores the stop script.

Users may see the following message in the service manager log:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked

Additionally, Users may see that every "'shutdown-cluster' invoked" message is immediately followed by a "Cluster shutdown already in progress!" message. For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

However, the shutdown does not proceed as expected.

Summary of symptoms:

Failed graceful shutdown for HA sites
Service Manager is unresponsive to the stop script, even with the -c option
Engines fail to promote to PRIMARY on a particular node (even if there are no other primaries and the engine is a RUNNING REPLICA)

Cause

Action

This issue can be remediated by restarting service manager by running the following:

<APPIAN_HOME>/services/stop.sh -p XXXXXXX -s manager

<APPIAN_HOME>/services/start.sh -p XXXXXXX -s all

Affected Versions

This article applies to all versions of Appian running with a High Availability Topology

Last Reviewed: May 2020

Tags: engines, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

Philip Moss — Mon, 04 May 2020 13:29:49 GMT

Revision 2 posted to Appian Knowledge Base by Philip Moss on 5/4/2020 1:29:49 PM

Symptoms

While attempting to shutdown the engines in an HA cluster, the service manager process refuses to carry out the shutdown and ignores the stop script.

Users may see the following message in the service manager log:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked

Additionally, Users may see that every "'shutdown-cluster' invoked" message is immediately followed by a "Cluster shutdown already in progress!" message. For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

However, the shutdown does not proceed as expected.

Summary of symptoms:

Failed graceful shutdown for HA sites
Service Manager is unresponsive to the stop script, even with the -c option
Engines fail to promote to PRIMARY on a particular node (even if there are no other primaries and the engine is a RUNNING REPLICA)

Cause

Action

This issue can be remediated by restarting service manager.

Affected Versions

This article applies to all versions of Appian running with a High Availability Topology

Last Reviewed: May 2020

Tags: engines, service manager

KB-XXXX High Availability Service Manager cluster refuses to shutdown

Sean Kim — Thu, 30 Apr 2020 13:35:06 GMT

Revision 1 posted to Appian Knowledge Base by Sean Kim on 4/30/2020 1:35:06 PM

Symptoms

While attempting to shutdown the engines in an HA cluster, the service manager process refuses to carry out the shutdown and ignores the stop script.

Users may see the following message in the logs:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked

However the shutdown does not proceed as expected.

Alternatively, if there was an issue with a graceful shutdown, the cluster-stop state is most likely the root cause if every "'shutdown-cluster' invoked" message is immediately followed by a "Cluster shutdown already in progress!" message in the SM log. For example:

INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Manager action 'shutdown-cluster' invoked
INFO  com.appian.komodo.admin.route.ManagerActionsRoute - Cluster shutdown already in progress!

Summary of symptoms:

Failed graceful shutdown for HA sites
Service Manager is unresponsive to the stop script, even with the -c option.
Engines fail to promote to PRIMARY on a particular node (even if there are no other primaries and the engine is a RUNNING REPLICA)

Cause

Typically, a user is trying to shutdown service manager on just one node in an HA configuration, but accidentally does a cluster stop which affects all other service managers in a cluster. SM could also get into this state if there is an issue with an intentional cluster shutdown, but only the engines are restarted and not service manager.

Action

This issue can be remediated by restarting service manager.

Affected Versions

This article applies to all versions of Appian running with a High Availability Topology

Last Reviewed: May 2020

Tags: engines, service manager