KB-2039 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

Parmida Borhani — Mon, 15 Jun 2020 01:18:52 GMT

Current Revision posted to Appian Knowledge Base by Parmida Borhani on 6/15/2020 1:18:52 AM

This issue has been resolved in an Appian hotfix/new Appian version. Please apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed and the following errors are seen in the application server log:

ERROR com.appiancorp.process.engine.StartProcessByEventRequest - Could not start process model [draftId=<ID>, version=<version>, userContext=<user>]
com.appian.komodo.api.exceptions.SignalException: domain

ERROR com.appiancorp.common.config.ConfigObject - An error occurred while trying to initialize the config LoadExceptionHandling [0ms] [resources 16ms] com.appian.komodo.api.exceptions.SignalException: domain at

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will roll over anytime an execution engine is restarted. Therefore, it has been calculated that this behavior should only occur if an execution engine has been running continuously for no less than approximately 60 days.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been addressed via AN-146311 in the following hotfixes/versions:

Action

Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Workaround

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

Self-managed

This issue can be resolved by restarting the environment. To avoid downtime in the environment, moving the above mentioned log files to a directory other than <APPIAN_HOME>/logs/perflogs and restarting the afflicted engine will resolve this issue. A single execution engine can be restarted with the following procedures:

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 to 19.4

Last Reviewed: June 2020

Tags: known issues, domain, 500, engines, fixed issues, Performance, infrastructure, top processes

KB-2039 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

Parmida Borhani — Mon, 15 Jun 2020 01:16:50 GMT

Revision 18 posted to Appian Knowledge Base by Parmida Borhani on 6/15/2020 1:16:50 AM

This issue has been resolved in an Appian hotfix/new Appian version. Please apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed and the following errors are seen in the application server log:

ERROR com.appiancorp.process.engine.StartProcessByEventRequest - Could not start process model [draftId=<ID>, version=<version>, userContext=<user>]
com.appian.komodo.api.exceptions.SignalException: domain

ERROR com.appiancorp.common.config.ConfigObject - An error occurred while trying to initialize the config LoadExceptionHandling [0ms] [resources 16ms] com.appian.komodo.api.exceptions.SignalException: domain at

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

Action

Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Workaround

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 to 19.4

Last Reviewed: June 2020

Tags: known issues, 500, engines, fixed issues, Performance, infrastructure

KB-2039 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

Parmida Borhani — Mon, 15 Jun 2020 01:15:11 GMT

Revision 17 posted to Appian Knowledge Base by Parmida Borhani on 6/15/2020 1:15:11 AM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed and the following errors are seen in the application server log:

ERROR com.appiancorp.process.engine.StartProcessByEventRequest - Could not start process model [draftId=<ID>, version=<version>, userContext=<user>]
com.appian.komodo.api.exceptions.SignalException: domain

ERROR com.appiancorp.common.config.ConfigObject - An error occurred while trying to initialize the config LoadExceptionHandling [0ms] [resources 16ms] com.appian.komodo.api.exceptions.SignalException: domain at

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

Action

Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Workaround

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 to 19.4

Last Reviewed: June 2020

Tags: known issues, 500, engines, Performance, infrastructure, open issues

KB-2039 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

Parmida Borhani — Mon, 18 May 2020 22:06:40 GMT

Revision 16 posted to Appian Knowledge Base by Parmida Borhani on 5/18/2020 10:06:40 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

Action

Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Workaround

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 to 19.4

Last Reviewed: April 2020

Tags: known issues, 500, engines, Performance, infrastructure, open issues

KB-2039 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

James Lee — Thu, 23 Apr 2020 18:29:06 GMT

Revision 15 posted to Appian Knowledge Base by James Lee on 4/23/2020 6:29:06 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

Action

Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Workaround

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 to 19.4

Last Reviewed: April 2020

Tags: known issues, engines, Performance, infrastructure, open issues

KB-2039 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

Rebecca Jonas — Tue, 10 Dec 2019 18:51:54 GMT

Revision 13 posted to Appian Knowledge Base by Rebecca Jonas on 12/10/2019 6:51:54 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, Performance, infrastructure, open issues

KB-2039 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

Rebecca Jonas — Tue, 10 Dec 2019 18:51:54 GMT

Revision 14 posted to Appian Knowledge Base by Rebecca Jonas on 12/10/2019 6:51:54 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, Performance, infrastructure, open issues

KB-XXXX 500 Internal Server Errors displayed when accessing some processes with single execution engine running at 100% CPU

James Lee — Fri, 06 Dec 2019 17:41:24 GMT

Revision 12 posted to Appian Knowledge Base by James Lee on 12/6/2019 5:41:24 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with this article.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the above steps fail or take an extended amount of time, follow the procedures below to forcibly kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, Performance, infrastructure, open issues

KB-XXXX 500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 23:04:08 GMT

Revision 11 posted to Appian Knowledge Base by James Lee on 12/4/2019 11:04:08 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will rollover anytime an execution engine is restarted. Therefore, it has been calculated that this behavior should only occur if an execution engine has been running continuously for no less than approximately 60 days.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

This issue can be resolved by rebooting the environment. To avoid downtime in the environment, moving the above mentioned log files to a directory other than <APPIAN_HOME>/logs/perflogs and restarting the afflicted engine will resolve this issue. A single execution engine can be restarted with the following procedures, but if this fails or takes an extended amount of time, there are further steps listed on how to forcibly kill the engine:

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the afflicted engine needs to be forcibly killed, follow the procedures below to kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, Performance, infrastructure, open issues

KB-XXXX 500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 16:55:26 GMT

Revision 10 posted to Appian Knowledge Base by James Lee on 12/4/2019 4:55:26 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will rollover anytime an execution engine is restarted. Therefore, it has been calculated that this behavior should only occur if an execution engine has been running continuously for no less than approximately 60 days.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

This issue can be resolved by rebooting the environment. To avoid downtime in the environment, moving the above mentioned log files to a directory other than <APPIAN_HOME>/logs/perflogs and restarting the afflicted engine will resolve this issue. A single execution engine can be restarted with the following procedures, but if this fails or takes an extended amount of time, there are further steps listed on how to forcibly kill the engine:

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the afflicted engine needs to be forcibly killed, follow the procedures below to kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, Performance, infrastructure, open issues

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 14:46:13 GMT

Revision 9 posted to Appian Knowledge Base by James Lee on 12/4/2019 2:46:13 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will rollover anytime an execution engine is restarted. Therefore, it has been calculated that this behavior should only occur if an execution engine has been running continuously for no less than approximately 60 days.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the afflicted engine needs to be forcibly killed, follow the procedures below to kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, Performance, infrastructure, open issues

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 14:45:24 GMT

Revision 8 posted to Appian Knowledge Base by James Lee on 12/4/2019 2:45:24 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will rollover anytime an execution engine is restarted. Therefore, it has been calculated that this behavior should only occur if an execution engine has been running continuously for no less than approximately 60 days.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the afflicted engine needs to be forcibly killed, follow the procedures below to kill and restart the engine:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, infrastructure, open issues

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 14:44:19 GMT

Revision 7 posted to Appian Knowledge Base by James Lee on 12/4/2019 2:44:19 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will rollover anytime an execution engine is restarted. Therefore, it has been calculated that this behavior should only occur if an execution engine has been running continuously for no less than approximately 60 days.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in these logs become too massive to scale any further, which causes the engine to lock while trying to process the data. This issue has been reported to the Appian Product Team, and the reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the afflicted engine needs to be forcibly killed, follow the procedures below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, infrastructure, open issues

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 14:42:15 GMT

Revision 6 posted to Appian Knowledge Base by James Lee on 12/4/2019 2:42:15 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will rollover anytime an execution engine is restarted, so this behavior should only manifest if an execution engine has been running for no less than approximately 60 days continuously.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in this log become too massive to scale any further, which causes the engine to lock trying to process further data. This issue has been reported to the Appian Product Team. The reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the afflicted engine needs to be forcibly killed, follow the procedures below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, infrastructure, open issues

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 14:40:26 GMT

Revision 5 posted to Appian Knowledge Base by James Lee on 12/4/2019 2:40:26 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage on the server hosting the Appian engines, a single execution engine is found to be running at 100% CPU. Steps to verify this are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

These logs can be found in the <APPIAN_HOME>/logs/perflogs directory. These logs will rollover anytime an execution engine is restarted, so this behavior will only manifest if an execution engine has been running for at least 60 days continuously.

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in this log become too massive to scale any further, which causes the engine to lock trying to process further data. This issue has been reported to the Appian Product Team. The reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

If the afflicted engine needs to be forcibly killed, follow the procedures below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, infrastructure, open issues

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 14:18:44 GMT

Revision 4 posted to Appian Knowledge Base by James Lee on 12/4/2019 2:18:44 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage, a single execution engine is found to be running at 100% CPU. Steps to verify are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in this log become too massive to scale any further, which causes the engine to lock trying to process further data. This issue has been reported to the Appian Product Team. The reference number for this issue is AN-146311.

Action

Cloud

Open a case with Appian Support and note to the case description that you are experiencing behavior in line with KB-XXXX.

On-Premise

This issue can be resolved by rebooting the environment. To avoid downtime in the environment, moving the above mentioned log files to a directory other than <APPIAN_HOME>/logs/perflogs and restarting the afflicted engine will resolve this issue. A single engine can be restarted with the following procedures:

Note: If your environment is configured for high availability, then a replica execution engine will automatically be promoted to primary when the afflicted engine is stopped or forcibly killed.

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

Note: the afflicted engine may need to be killed forcibly, and this can be done using the steps below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX to restart the engine.

Affected Versions

This article applies to Appian 18.3 and later.

Last Reviewed: December 2019

Tags: known issues, engines, infrastructure, open issues

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 14:08:09 GMT

Revision 3 posted to Appian Knowledge Base by James Lee on 12/4/2019 2:08:09 PM

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage, a single execution engine is found to be running at 100% CPU. Steps to verify are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in this log become too massive to scale any further, which causes the engine to lock trying to process further data. This issue has been reported to the Appian Product Team. The reference number for this issue is AN-146311.

Action

To avoid downtime in the environment, moving the above mentioned log files to a directory other than <APPIAN_HOME>/logs/perflogs and restarting the afflicted engine will resolve this issue. A single engine can be restarted with the following procedures:

From <APPIAN_HOME>/services/bin, execute ./stop.sh (.bat) -p <PASSWORD> -s executionXX. Wait for the engine to stop.
From <APPIAN_HOME>/services/bin, execute ./start.sh (.bat) -p <PASSWORD> -s executionXX.

The afflicted engine may need to be killed forcibly, and this can be done using the steps below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the newly selected PID column.
You can force kill the engine by right clicking the engine process in task manager, and selecting the option to "End task."

This issue can also be resolved by rebooting the environment.

Affected Versions

This section includes any relevant version information for Appian or other third/party configurations. Some examples of valid affected versions are as follows:

This article applies to all versions of Appian.
This article applies to Appian 7.11 and later.
This article applies to Appian 16.2 and earlier.
This article applies to Appian 7.11 and 16.1.
This article applies to Appian 16.1, 16.2, and 16.3.
This article applies to all versions of Appian from Appian 7.10 to Appian 16.2.
This article applies to all versions of Appian using JBoss EAP 6.4.9 as an application server and Internet Explorer 9 as a web browser.
This article applies to all versions of Appian Cloud.

Last Reviewed: Month YYYY

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 13:48:40 GMT

Revision 2 posted to Appian Knowledge Base by James Lee on 12/4/2019 1:48:40 PM

This issue has been resolved in an Appian hotfix/new Appian version. Please apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage, a single execution engine is found to be running at 100% CPU. Steps to verify are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This issue occurs when one of the below engine performance logs reaches 10 megabytes in size:

execution_by_category_*
top_models_by_time_*
top_processes_by_time_*

The reason these logs can cause an execution engine to max out at 100% CPU utilization is a result of how select values in these logs are calculated. Transient tables within the execution engine that populate values in this log become too massive to scale any further, which causes the engine to lock trying to process further data. The reference number for this issue is AN-146311.

Action

To avoid downtime in the environment, moving the above mentioned log files to a directory other than <APPIAN_HOME>/logs/perflogs and restarting the afflicted engine will resolve this issue. The afflicted engine may need to be killed forcibly, and this can be done using the steps below:

Linux

Similar to the steps listed in the "Cause" section of this article:

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for the execution engine process running at 100% CPU. The PID of this process will be listed in the left most column of the top output.
You can use the following command to forcibly kill the engine: kill -9 <PID>

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "PID" and "Command line" columns:

Workaround

This section is optional and should only be used as an alternative action if there is a massive technical debt associated with performing the action in the above section for customers (such as upgrading). It can also be used if there is a solution to the issue in the article but is more inconvenient to implement or it causes other undesirable behavior and wouldn’t warrant being an action.

Affected Versions

This section includes any relevant version information for Appian or other third/party configurations. Some examples of valid affected versions are as follows:

This article applies to all versions of Appian.
This article applies to Appian 7.11 and later.
This article applies to Appian 16.2 and earlier.
This article applies to Appian 7.11 and 16.1.
This article applies to Appian 16.1, 16.2, and 16.3.
This article applies to all versions of Appian from Appian 7.10 to Appian 16.2.
This article applies to all versions of Appian using JBoss EAP 6.4.9 as an application server and Internet Explorer 9 as a web browser.
This article applies to all versions of Appian Cloud.

Last Reviewed: Month YYYY

500 Internal Server Errors displayed when accessing some processes. Single execution engine running at 100% CPU

James Lee — Wed, 04 Dec 2019 13:23:30 GMT

Revision 1 posted to Appian Knowledge Base by James Lee on 12/4/2019 1:23:30 PM

This issue has been resolved in an Appian hotfix/new Appian version. Please apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

Symptoms

When trying to initiate some processes in an environment, a 500 - Internal Server Error is displayed.

When checking CPU usage, a single execution engine is found to be running at 100% CPU. Steps to verify are listed below:

Linux

Run the top command on the server, and then press c, shift+p. This will sort processes by CPU usage.
Under the CPU column, look for an execution engine process running at 100% CPU.
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Windows

Open up Task Manager and navigate to the "Processes" tab.
Right click on the "Name" column header, and select the option to add the "Command line" column:
Under the CPU column, look for an execution engine process running at 100% CPU (you will likely need to expand the "Command line" column in order to see the full string).
1. You can identify an engine process if it contains the string <APPIAN_HOME>/server/_bin/k/linux64/k <APPIAN_HOME>/server/_lib/adb
2. You can identify the specific engine as an execution engine by looking at the end part of the string. For example, <APPIAN_HOME>/server/process/exec/02/gw1 would indicate that the engine is execution02.

Cause

This section is required for this template. When writing a cause for a particular article, ensure the following is taken into consideration:

The cause is concise and does not disclose more information than absolutely necessary.
The cause does not share any Appian source code. Source code for plugins/shared components are fine.
If this is an issue outside of the Appian product and the third-party vendor included documentation referencing the bug, provide their documentation for reference.

For any enhancements/defects, use one of the following as applicable:

This issue has been reported to the Appian Product Team. The reference number for this issue is AN-XXXXX.
This product use case has been logged to the Appian Product Team for consideration to add this functionality in the product. Kindly note it is not Appian Support’s policy to disclose how or when a product use case will be implemented.
This issue has been addressed via AN-XXXXX in Appian X.X Hotfix Package X.
This issue has been addressed via AN-XXXXX in the following hotfixes/versions:
- List each specific hotfix package/version in ascending order by version.
This use case has been incorporated in Appian X.X Hotfix Package X.
This use case has been incorporated in the following versions/hotfixes:
- List each specific hotfix package/version in ascending order by version.

Note: When listing hotfixes, add a hyperlink to the Hotfix documentation for that version.

When logging a product defect, ensure you use the known issues tag as well as the open issues or fixed issues tag, depending if the issue is fixed or not.

If no cause is known and this is not something Engineering can reproduce/needs to know about, then say “The root cause of this issue is currently unknown.” If you use that as your cause, then a valid action must be present.

Action

This section is required for this template and contains the steps to resolve the issue. If an issue in an article is solved by a hotfix and/or major version of Appian, use the following language to indicate that:

Apply the latest hotfix to your Appian installation.
Upgrade to the latest version of Appian.
Apply the latest hotfix to your Appian installation or upgrade to the latest version of Appian.

If there is a product issue with Engineering that is open and a workaround for that issue exists, include it in the Action section until the fix is available. After the fix is made available, use that as a workaround instead.

Otherwise, the content in this section should wholly solve a customer’s issue without producing any undesirable side effects. If there is at least one valid action that doesn’t cause undesirable side effects, all other actions with negative side effects should go in the Workaround section.

If there is no action or workaround that resolves the issue, use the following language:

There is no action or workaround currently available for this issue. If you are facing this issue, please open a support case with Appian.

Workaround

Affected Versions

This section includes any relevant version information for Appian or other third/party configurations. Some examples of valid affected versions are as follows:

This article applies to all versions of Appian.
This article applies to Appian 7.11 and later.
This article applies to Appian 16.2 and earlier.
This article applies to Appian 7.11 and 16.1.
This article applies to Appian 16.1, 16.2, and 16.3.
This article applies to all versions of Appian from Appian 7.10 to Appian 16.2.
This article applies to all versions of Appian using JBoss EAP 6.4.9 as an application server and Internet Explorer 9 as a web browser.
This article applies to all versions of Appian Cloud.

Last Reviewed: Month YYYY