KB-XXXX Enabling automatic heap dumps on OutOfMemoryError for Appian Search Server

Symptom

The Appian Search Server may on rare occasions crash due to an OutOfMemoryError. This may either happen only once or intermittently, and the root cause for these events may vary.

Cause

The Appian Search Server crashing with an OutOfMemoryError indicates that the search server has run out of available heap space. Default heap space for the Search Server is 1 gigabyte. For production servers with large search indices, this may need to be increased, but prior to increasing heap space it is necessary to investigate why the issue is occurring in order to ensure system stability.

Action

In order to investigate the root cause of the OutOfMemoryError, heap dumps must be automatically gathered from the time this error occurs. The heap dump contains the objects stored by the JVM instance on which Appian Search Server runs and will assist Appian Technical Support in determining the root cause behind the issue. In order to enable automatic heap dumps, amend the start.bat|sh script located in <APPIAN_HOME>/search-server/bin to include the flag to automatically capture a heap dump:

1. Add the -XX:+HeapDumpOnOutOfMemoryError argument to the search-server JVM arguments so it can create a heap dump if it runs out of heap space.
   1. Open the start.bat|sh script with a text editor
   2. Locate the following line:
      

      SS_MEM_ARGS="-Xms1024m -Xmx1024m"

   3. Alter it such that the line reads:
      

      SS_MEM_ARGS="-Xms1024m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError"

   4. Save the file
2. Restart the search server and tail the log. Ascertain that the startup output contains the following lines:
   

   [INFO ][com.appian.es.bootstrap.NodeInit] JVM Arguments:
   
     -Xms1024m
   
     -Xmx1024m
   
     -XX:+HeapDumpOnOutOfMemoryError

Affected versions

This article applies to all versions of Appian.