KB-2231 Unable to shut down Appian Java components with OOTB stop script

Maggie Deppe-Walker — Wed, 01 Feb 2023 22:49:05 GMT

Current Revision posted to Appian Knowledge Base by Maggie Deppe-Walker on 2/1/2023 10:49:05 PM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's out-of-the-box (OOTB) stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/.
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat.

Cause

The stop.sh and stop-appserver.sh script searches for Zookeeper, Kafka, and Tomcat in Java processes, which are referred to in the jps command. jps searches /tmp/hsperfdata_<current_user>, where it expects Java processes to have created a directory with metadata.

When the /tmp/hsperfdata_<current_user> has a file for each of the Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

When the /tmp/hsperfdata_<current_user> does not contain a file for each of the Appian Java process' PID, the corresponding process will not show up in the output of jps command, resulting in the failure of the OOTB stop scripts to stop Appian Java components.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat

If Tomcat cannot be stopped with the OOTB stop-appserver script, follow the below steps to bring down tomcat:

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper

If Kafka and Zookeeper cannot be stopped with the OOTB stop script, follow the below steps to bring down Kafka and Zookeeper:

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper on all nodes hosting Kafka and Zookeeper to identify the PIDs respectively.
Manually terminate Kafka on all nodes hosting Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper on all nodes hosting Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down on all nodes.
Start up Internal Messaging Services using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script.
Stop all the components gracefully depending on the installation type listed below
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: January 2023

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Maggie Deppe-Walker — Thu, 05 Jan 2023 03:48:59 GMT

Revision 31 posted to Appian Knowledge Base by Maggie Deppe-Walker on 1/5/2023 3:48:59 AM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's out-of-the-box (OOTB) stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/.
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat.

Cause

When the /tmp/hsperfdata_<current_user> has a file for each of the Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat

If Tomcat cannot be stopped with the OOTB stop-appserver script, follow the below steps to bring down tomcat:

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper

If Kafka and Zookeeper cannot be stopped with the OOTB stop script, follow the below steps to bring down Kafka and Zookeeper:

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper on all nodes hosting Kafka and Zookeeper to identify the PIDs respectively.
Manually terminate Kafka on all nodes hosting Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper on all nodes hosting Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down on all nodes.
Start up Internal Messaging Services using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script.
Stop all the components gracefully depending on the installation type listed below
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Maggie Deppe-Walker — Thu, 05 Jan 2023 03:47:58 GMT

Revision 30 posted to Appian Knowledge Base by Maggie Deppe-Walker on 1/5/2023 3:47:58 AM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's out-of-the-box (OOTB) stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/.
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat.

Cause

When the /tmp/hsperfdata_<current_user> has a file for each of the Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat

If Tomcat cannot be stopped with the OOTB stop-appserver script, follow the below steps to bring down tomcat

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper

If Kafka and Zookeeper cannot be stopped with the OOTB stop script, follow the below steps to bring down tomcat

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper on all nodes hosting Kafka and Zookeeper to identify the PIDs respectively.
Manually terminate Kafka on all nodes hosting Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper on all nodes hosting Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down on all nodes.
Start up Internal Messaging Services using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script.
Stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Maggie Deppe-Walker — Thu, 05 Jan 2023 03:45:45 GMT

Revision 29 posted to Appian Knowledge Base by Maggie Deppe-Walker on 1/5/2023 3:45:45 AM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/.
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat.

Cause

When the /tmp/hsperfdata_<current_user> has a file for each of the Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat if Tomcat cannot be stopped with the OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with the OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper on all nodes hosting Kafka and Zookeeper to identify the PIDs respectively.
Manually terminate Kafka on all nodes hosting Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper on all nodes hosting Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down on all nodes.
Start up Internal Messaging Services using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script.
Stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Maggie Deppe-Walker — Wed, 04 Jan 2023 01:23:41 GMT

Revision 28 posted to Appian Knowledge Base by Maggie Deppe-Walker on 1/4/2023 1:23:41 AM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/.
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat.

Cause

When the /tmp/hsperfdata_<current_user> has a file for each Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

When the /tmp/hsperfdata_<current_user> does not contain a file for the Appian Java process' PID, then the corresponding process will not show up in the output of jps command, resulting in the failure of stopping the Appian Java components with OOTB stop scripts.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat if Tomcat cannot be stopped with OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper on all nodes hosting Kafka and Zookeeper to identify the PIDs respectively.
Manually terminate Kafka on all nodes hosting Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper on all nodes hosting Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down on all nodes.
Start up Internal Messaging Services using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script.
Stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Tue, 03 Jan 2023 18:59:57 GMT

Revision 27 posted to Appian Knowledge Base by Yiyi Tao on 1/3/2023 6:59:57 PM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat

Cause

When the /tmp/hsperfdata_<current_user> has a file for each Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat if Tomcat cannot be stopped with OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper on all nodes hosting Kafka and Zookeeper to identify the PIDs respectively.
Manually terminate Kafka on all nodes hosting Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper on all nodes hosting Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down on all nodes.
Start up Internal Messaging Services using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script. Then stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Tue, 03 Jan 2023 18:58:36 GMT

Revision 26 posted to Appian Knowledge Base by Yiyi Tao on 1/3/2023 6:58:36 PM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat

Cause

When the /tmp/hsperfdata_<current_user> has a file for each Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat if Tomcat cannot be stopped with OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper on all nodes hosting Zookeeper and Kafka to identify the PIDs for Kafka and Zookeeper respectively.
Manually terminate Kafka on all nodes hosting Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper on all nodes hosting Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down on all nodes.
Start up Internal Messaging Services using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script. Then stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Tue, 03 Jan 2023 18:57:08 GMT

Revision 25 posted to Appian Knowledge Base by Yiyi Tao on 1/3/2023 6:57:08 PM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat

Cause

When the /tmp/hsperfdata_<current_user> has a file for each Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Bring down Tomcat if Tomcat cannot be stopped with OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively.
Manually terminate Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down.
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script. Then stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Tue, 03 Jan 2023 18:56:01 GMT

Revision 24 posted to Appian Knowledge Base by Yiyi Tao on 1/3/2023 6:56:01 PM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat

Cause

When the /tmp/hsperfdata_<current_user> has a file for each Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

How to bring down Tomcat if Tomcat cannot be stopped with OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

How to bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively.
Manually terminate Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down.
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script. Then stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Tue, 03 Jan 2023 18:55:36 GMT

Revision 23 posted to Appian Knowledge Base by Yiyi Tao on 1/3/2023 6:55:36 PM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat

Cause

When the /tmp/hsperfdata_<current_user> has a file for each Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

How to bring down Tomcat if Tomcat cannot be stopped with OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

How to bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively.
Manually terminate Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down.
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script. Then use the stop all the components gracefully:
1. Standalone: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> -s all on the node hosting Zookeeper and Kafka.
2. HA: run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Tue, 03 Jan 2023 18:53:44 GMT

Revision 22 posted to Appian Knowledge Base by Yiyi Tao on 1/3/2023 6:53:44 PM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of jps command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat

Cause

When the /tmp/hsperfdata_<current_user> has a file for each Appian Java process' PID, the corresponding process will show up in the jps command output and the OOTB stop scripts should work.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the jps command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

How to bring down Tomcat if Tomcat cannot be stopped with OOTB stop-appserver script

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

How to bring down Kafka and Zookeeper if Kafka and Zookeeper cannot be stopped with OOTB stop script

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively.
Manually terminate Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down.
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s manager on all nodes hosting Kafka and Zookeeper. This should start up Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the jps command should return PIDs for Service Manager, Kafka and Zookeeper.
Confirm that Zookeeper cluster and Kafka cluster are up and running using status script, use the stop script in <APPIAN_HOME>/services/bin to stop all the components gracefully - In standalone setup, run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> -s all on the node hosting Zookeeper and Kafka. In HA setup, run <APPIAN_HOME>/services/bin/stop.sh -p <service_manager_password> on all nodes hosting Zookeeper and Kafka.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Maggie Deppe-Walker — Thu, 29 Dec 2022 03:04:12 GMT

Revision 21 posted to Appian Knowledge Base by Maggie Deppe-Walker on 12/29/2022 3:04:12 AM

Symptoms

Zookeeper, Kafka, and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The process IDs (PIDs) of Zookeeper, Kafka, and/or Tomcat are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain the PIDs of Zookeeper, Kafka, and/or Tomcat

Cause

The stop.sh and stop-appserver.sh script searches for Zookeeper, Kafka, and Tomcat in Java processes, which are referred to in the JPS command. JPS searches /tmp/hsperfdata_<current_user>, where it expects Java processes to have created a directory with metadata.

When the /tmp/hsperfdata_<current_user> has a file for the Appian Java process' PID, the corresponding process will show up in the JPS command output and the OOTB stop scripts should work.

When the /tmp/hsperfdata_<current_user> does not contain a file for the Appian Java process' PID, then the corresponding process will not show up in the output of JPS command, resulting in the failure of stopping the Appian Java components with OOTB stop scripts.

Action

Confirm if the stop script is running as a different user than the Appian Java process, as the directory that the JPS command searches in is dependent on the current user. If the user is different, make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or the entire /tmp directory.

Workaround

Manually bring down Tomcat

Run ps -ef | grep tomcat to identify the PID for Tomcat.
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Gracefully bring down Kafka and Zookeeper

Make sure app server(s), search server(s), data server(s), and engines have all been stopped.
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively.
Manually terminate Kafka using the following command: kill -9 <Kafka_PID>
Next, manually terminate Zookeeper using the following command: kill -9 <Zookeeper_PID>
Confirm all Appian components are down.
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result the JPS command should return PIDs for Service Manager, Kafka and Zookeeper.
Run the stop script in <APPIAN_HOME>/services/bin to stop all the components gracefully.

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Fri, 16 Dec 2022 18:16:02 GMT

Revision 20 posted to Appian Knowledge Base by Yiyi Tao on 12/16/2022 6:16:02 PM

Symptoms

Customer observes that Zookeeper, Kafka and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The PIDs of Zookeeper, Kafka and/or Tomcat are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Zookeeper, Kafka and/or Tomcat

Cause

The stop.sh and stop-appserver.sh script searches for Zookeeper, Kafka and Tomcat in java processes, which are referring to JPS. JPS searches /tmp/hsperfdata_<current_user>, where it expects java processes to have created a directory with some metadata. When the /tmp/hsperfdata_<current_user> has a file for the Appian Java process' PID, the corresponding process will show up in the JPS command output and the stop script would work. When the /tmp/hsperfdata_<current_user> does not contain a file for the Appian Java process' PID, then the corresponding process will not show up in the output of JPS command, resulting in the failure of stopping the Appian Java components with OOTB stop scripts.

Action

Please confirm if the stop script is running as a different user than the Appian Java process as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see below to bring down Tomcat when the issue occurs:

Run ps -ef | grep tomcat to identify the PID for Tomcat
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Please see below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines have all been stopped
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively
Manually terminate Kafka first then Zookeeper by using kill -9 <Kafka_PID> and kill -9 <Zookeeper_PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Fri, 16 Dec 2022 05:16:46 GMT

Revision 19 posted to Appian Knowledge Base by Yiyi Tao on 12/16/2022 5:16:46 AM

Symptoms

Customer observes that Zookeeper, Kafka and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The PIDs of Zookeeper, Kafka and/or Tomcat are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Zookeeper, Kafka and/or Tomcat

Cause

Action

Please confirm if the stop script is running as a different user than the Appian Java process as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see below to bring down Tomcat when the issue occurs:

Run ps -ef | grep tomcat to identify the PID for Tomcat
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Please see below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines have all been stopped
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively
Manually terminate Kafka first then Zookeeper by using kill -9 <Kafka_PID> and kill -9 <Zookeeper_PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Appian Java components with OOTB stop script

Yiyi Tao — Fri, 16 Dec 2022 05:15:47 GMT

Revision 18 posted to Appian Knowledge Base by Yiyi Tao on 12/16/2022 5:15:47 AM

Symptoms

Customer observes that Zookeeper, Kafka and/or Tomcat intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin/ or stop-appserver.sh in <APPIAN_HOME>/tomcat/apache-tomcat/bin/
The PIDs of Zookeeper, Kafka and/or Tomcat are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Zookeeper, Kafka and/or Tomcat

Cause

Action

Please confirm if the stop script is running as a different user than the Appian Java process as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as the Appian Java process in question.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see the workaround step below to bring down Tomcat when the issue occurs:

Run ps -ef | grep tomcat to identify the PID for Tomcat
Manually terminate Tomcat by using kill -9 <Tomcat_PID>

Please see the workaround steps below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines have all been stopped
Run ps -ef | grep kafka and ps -ef | grep zookeeper to identify the PIDs for Kafka and Zookeeper respectively
Manually terminate Kafka first then Zookeeper by using kill -9 <Kafka_PID> and kill -9 <Zookeeper_PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Kafka and Zookeeper with OOTB stop script

Yiyi Tao — Tue, 13 Dec 2022 14:12:48 GMT

Revision 17 posted to Appian Knowledge Base by Yiyi Tao on 12/13/2022 2:12:48 PM

Symptoms

Customer observes that Kafka and Zookeeper intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin
The PIDs of Kafka and Zookeeper are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Kafka and Zookeeper

Cause

The stop script searches for Kafka and Zookeeper in java processes, which are referring to JPS. JPS searches /tmp/hsperfdata_<current_user>, where it expects java processes to have created a directory with some metadata. When the /tmp/hsperfdata_<current_user> has a file for Kafka and/or Zookeeper's PID, the corresponding process will show up in the JPS command output and the stop script would work. When the /tmp/hsperfdata_<current_user> does not contain a file for Kafka and Zookeeper's PID, then the corresponding processes will not show up in the output of JPS command, resulting in the failure of stopping Kafka and Zookeeper with stop.sh script in <APPIAN_HOME>/services/bin

Action

Please confirm if the stop script is running as a different user than the Kafka and Zookeeper processes as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as Kafka and Zookeeper.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see the workaround steps below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines are all stopped in the recommended way
Run ps -ef | grep java to identify the PIDs for Kafka and Zookeeper
Manually terminate Kafka first then Zookeeper by using kill -9 <PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Kafka and Zookeeper with OOTB stop script

Yiyi Tao — Tue, 13 Dec 2022 14:12:16 GMT

Revision 16 posted to Appian Knowledge Base by Yiyi Tao on 12/13/2022 2:12:16 PM

Symptoms

Customer observes that Kafka and Zookeeper intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin
The PIDs of Kafka and Zookeeper are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Kafka and Zookeeper

Cause

Action

Please confirm if the stop script is running as a different user than the Kafka and Zookeeper processes as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as Kafka and Zookeeper.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see the workaround steps below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines are all stopped in the recommended way
Run ps -ef | grep java to identify the PIDs for Kafka and Zookeeper
Manually terminate Kafka first then Zookeeper by using kill -9 <PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Kafka and Zookeeper with OOTB stop script

Yiyi Tao — Tue, 13 Dec 2022 14:11:34 GMT

Revision 15 posted to Appian Knowledge Base by Yiyi Tao on 12/13/2022 2:11:34 PM

Symptoms

Customer observes that Kafka and Zookeeper intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin
The PIDs of Kafka and Zookeeper are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Kafka and Zookeeper

Cause

Action

Please confirm if the stop script is running as a different user than the Kafka and Zookeeper processes as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as Kafka and Zookeeper.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see the workaround steps below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines are all stopped in the recommended way
Run ps -ef | grep java to identify the PIDs for Kafka and Zookeeper
Manually terminate Kafka first then Zookeeper by using kill -9 <PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Kafka and Zookeeper with OOTB stop script

Yiyi Tao — Wed, 07 Dec 2022 15:26:19 GMT

Revision 14 posted to Appian Knowledge Base by Yiyi Tao on 12/7/2022 3:26:19 PM

Symptoms

Customer observes that Kafka and Zookeeper intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin
The PIDs of Kafka and Zookeeper are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Kafka and Zookeeper

Cause

Action

Please confirm if the stop script is running as a different user than the Kafka and Zookeeper processes as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as Kafka and Zookeeper.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see the workaround steps below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines are all stopped in the recommended way
Run ps -ef | grep java to identify the PIDs for Kafka and Zookeeper
Manually terminate Kafka first then Zookeeper by using kill <PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Kafka and Zookeeper with OOTB stop script

Yiyi Tao — Wed, 07 Dec 2022 15:22:50 GMT

Revision 13 posted to Appian Knowledge Base by Yiyi Tao on 12/7/2022 3:22:50 PM

Symptoms

Customer observe that Kafka and Zookeeper intermittently are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin
The PIDs of Kafka and Zookeeper are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Kafka and Zookeeper

Cause

Action

Please confirm if the stop script is running as a different user than the Kafka and Zookeeper processes as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as Kafka and Zookeeper.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see the workaround steps below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines are all stopped in the recommended way
Run ps -ef | grep java to identify the PIDs for Kafka and Zookeeper
Manually terminate Kafka first then Zookeeper by using kill <PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure

DRAFT SP-8038 Unable to shut down Kafka and Zookeeper with OOTB stop script

Yiyi Tao — Wed, 07 Dec 2022 05:57:03 GMT

Revision 12 posted to Appian Knowledge Base by Yiyi Tao on 12/7/2022 5:57:03 AM

Symptoms

Intermittently, Kafka and Zookeeper are not able to be stopped by Appian's OOTB stop.sh script in <APPIAN_HOME>/services/bin
The PIDs of Kafka and Zookeeper are not shown in the output of JPS command, but able to be identified in the output of ps -ef | grep java command.
/tmp/hsperfdata_<current_user> does not contain PIDs of Kafka and Zookeeper

Cause

Action

Please confirm if the stop script is running as a different user than the Kafka and Zookeeper processes as the directory JPS command searches in is dependent on the current user. If so, please make sure the stop script is ran by the same user as Kafka and Zookeeper.
If the stop script is ran by the exact same user as the running Java processes and the issue only happens after Appian runs for more than a certain period of time, please confirm with your server team to see if there is any cronjob/mechanism that is cleaning up the /tmp/hsperfdata_<current_user> folder or just /tmp in general.

Workaround

Please see the workaround steps below to bring down Kafka and Zookeeper gracefully when the issue occurs:

Make sure app server(s), search server(s), data server(s) and engines are all stopped in the recommended way
Run ps -ef | grep java to identify the PIDs for Kafka and Zookeeper
Manually terminate Kafka first then Zookeeper by using kill <PID>
Confirm all Appian components are down
Start up engines using <APPIAN_HOME>/services/bin/start.sh -p <password> -s all. This should start up Engines, Service Manager, Kafka, and Zookeeper. This should also generate the expected data in /tmp/hsperfdata_<current_user> and as a result JPS command should return PIDs for Service Manager, Kafka, and Zookeeper
Run stop script in <APPIAN_HOME>/services/bin as normal. This time the script should be able to stop all components gracefully

Affected Versions

This article applies to all self-managed versions of Appian on Linux.

Last Reviewed: December 2022

Tags: linux, infrastructure