KB-1446 How to enable automatic heap dumps on OutOfMemoryError for Appian Search Server

Parmida Borhani — Tue, 09 Jan 2018 01:38:09 GMT

Current Revision posted to Appian Knowledge Base by Parmida Borhani on 1/9/2018 1:38:09 AM

Purpose

The Appian Search Server may crash due to an OutOfMemoryError. An OutOfMemoryError indicates that the search server has run out of available heap space. The root cause for running out of heap space may vary.

Default heap space for the Search Server is 1GB. For production servers with large search indices, this may need to be increased, but prior to increasing heap space it is necessary to investigate why the issue is occurring in order to ensure system stability.

Instructions

In order to investigate the root cause of the OutOfMemoryError, a heap dump must be gathered from the time the heap usage was high. The heap dump contains a snapshot of the objects stored in memory by the Search Server and will assist Appian Technical Support in determining the root cause behind the issue.

In order to configure the Search Server to attempt to capture a heap dump when an OutOfMemoryError is triggered, amend the start.bat|sh script located in <APPIAN_HOME>/search-server/bin to include the flag to automatically capture a heap dump:

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor.
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file.
Restart the search server and review <APPIAN_HOME>/logs/search-server/search-server.log. Ensure that the startup output contains the following lines:
```
[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:
  -Xms1024m
  -Xmx1024m
  -XX:+HeapDumpOnOutOfMemoryError
```
If an OutOfMemoryError is detected, the Search Server will attempt to capture a heap dump under <APPIAN_HOME>/search-server/bin.

For information on how to analyze a heap dump, refer to KB-1176.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: January 2018

Tags: administration, heap, how-to, search server, scripts

DRAFT KB-XXXX How to enable automatic heap dumps on OutOfMemoryError for Appian Search Server

Parmida Borhani — Mon, 08 Jan 2018 20:37:01 GMT

Revision 7 posted to Appian Knowledge Base by Parmida Borhani on 1/8/2018 8:37:01 PM

Purpose

Instructions

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor.
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file.
Restart the search server and review <APPIAN_HOME>/logs/search-server/search-server.log. Ensure that the startup output contains the following lines:
```
[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:
  -Xms1024m
  -Xmx1024m
  -XX:+HeapDumpOnOutOfMemoryError
```
If an OutOfMemoryError is detected, the Search Server will attempt to capture a heap dump under <APPIAN_HOME>/search-server/bin.

For information on how to analyze a heap dump, refer to KB-1176.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: January 2018

Tags: administration, heap, how-to, search server, scripts

DRAFT KB-XXXX How to enable automatic heap dumps on OutOfMemoryError for Appian Search Server

Parmida Borhani — Mon, 08 Jan 2018 20:36:26 GMT

Revision 6 posted to Appian Knowledge Base by Parmida Borhani on 1/8/2018 8:36:26 PM

Purpose

Instructions

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor.
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file.
Restart the search server and review <APPIAN_HOME>/logs/search-server/search-server.log. Ensure that the startup output contains the following lines:
```
[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:
  -Xms1024m
  -Xmx1024m
  -XX:+HeapDumpOnOutOfMemoryError
```
If an OutOfMemoryError is detected, the Search Server will attempt to capture a heap dump under <APPIAN_HOME>/search-server/bin.

For information on how to analyze a heap dump, refer to KB-1176.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: January 2018

Tags: administration, heap, how-to, search server, scripts

DRAFT KB-XXXX How to enable automatic heap dumps on OutOfMemoryError for Appian Search Server

Tom Ryan — Mon, 08 Jan 2018 19:27:55 GMT

Revision 5 posted to Appian Knowledge Base by Tom Ryan on 1/8/2018 7:27:55 PM

Purpose

Instructions

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file
Restart the search server and review <APPIAN_HOME>/logs/search-server/search-server.log. Ensure that the startup output contains the following lines:
```
[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:
  -Xms1024m
  -Xmx1024m
  -XX:+HeapDumpOnOutOfMemoryError
```
If an OutOfMemoryError is detected, the Search Server will attempt to capture a heap dump under <APPIAN_HOME>/search-server/bin.

For information on how to analyze a heap dump, refer to KB-1176

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: January 2018

Tags: administration, heap, how-to, search server, scripts

KB-XXXX Enabling automatic heap dumps on OutOfMemoryError for Appian Search Server

Jussi Lundstedt — Mon, 08 Jan 2018 18:03:34 GMT

Revision 1 posted to Appian Knowledge Base by Jussi Lundstedt on 1/8/2018 6:03:34 PM

Symptom

The Appian Search Server may on rare occasions crash due to an OutOfMemoryError. This may either happen only once or intermittently, and the root cause for these events may vary.

Cause

The Appian Search Server crashing with an OutOfMemoryError indicates that the search server has run out of available heap space. Default heap space for the Search Server is 1 gigabyte. For production servers with large search indices, this may need to be increased, but prior to increasing heap space it is necessary to investigate why the issue is occurring in order to ensure system stability.

Action

In order to investigate the root cause of the OutOfMemoryError, heap dumps must be automatically gathered from the time this error occurs. The heap dump contains the objects stored by the JVM instance on which Appian Search Server runs and will assist Appian Technical Support in determining the root cause behind the issue. In order to enable automatic heap dumps, amend the start.bat|sh script located in <APPIAN_HOME>/search-server/bin to include the flag to automatically capture a heap dump:

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file

Restart the search server and tail the log. Ascertain that the startup output contains the following lines:

[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:

  -Xms1024m

  -Xmx1024m

  -XX:+HeapDumpOnOutOfMemoryError

Affected versions

This article applies to all versions of Appian.

Tags: administration, heap, search server, scripts

DRAFT KB-XXXX How to enable automatic heap dumps on OutOfMemoryError for Appian Search Server

Parmida Borhani — Mon, 08 Jan 2018 17:42:10 GMT

Revision 4 posted to Appian Knowledge Base by Parmida Borhani on 1/8/2018 5:42:10 PM

Purpose

The Appian Search Server may on rare occasions crash due to an OutOfMemoryError. This may either happen only once or intermittently, and the root cause for these events may vary.

Instructions

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file

Restart the search server and tail the log. Ensure that the startup output contains the following lines:

[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:
  -Xms1024m
  -Xmx1024m
  -XX:+HeapDumpOnOutOfMemoryError

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: January 2018

Tags: administration, heap, how-to, search server, scripts

KB-XXXX Enabling automatic heap dumps on OutOfMemoryError for Appian Search Server

Jussi Lundstedt — Mon, 08 Jan 2018 13:06:08 GMT

Revision 3 posted to Appian Knowledge Base by Jussi Lundstedt on 1/8/2018 1:06:08 PM

Symptom

The Appian Search Server may on rare occasions crash due to an OutOfMemoryError. This may either happen only once or intermittently, and the root cause for these events may vary.

Cause

Action

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file

Restart the search server and tail the log. Ascertain that the startup output contains the following lines:

[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:

  -Xms1024m

  -Xmx1024m

  -XX:+HeapDumpOnOutOfMemoryError

Affected versions

This article applies to all versions of Appian.

Tags: administration, heap, search server, scripts

KB-XXXX Enabling automatic heap dumps on OutOfMemoryError for Appian Search Server

Jussi Lundstedt — Mon, 08 Jan 2018 13:05:18 GMT

Revision 2 posted to Appian Knowledge Base by Jussi Lundstedt on 1/8/2018 1:05:18 PM

Symptom

The Appian Search Server may on rare occasions crash due to an OutOfMemoryError. This may either happen only once or intermittently, and the root cause for these events may vary.

Cause

Action

Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
1. Open the start.bat|sh script with a text editor
2. Locate the following line:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m"
```
3. Alter it such that the line reads:
```
SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"
```
4. Save the file

Restart the search server and tail the log. Ascertain that the startup output contains the following lines:

[INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:

  -Xms1024m

  -Xmx1024m

  -XX:+HeapDumpOnOutOfMemoryError

Affected versions

This article applies to all versions of Appian.

Tags: administration, heap, search server, scripts