KB-1766 "High disk watermark [90%] exceeded" warning seen in the search-server logs

Rebecca Jonas — Mon, 27 Jan 2020 18:27:26 GMT

Current Revision posted to Appian Knowledge Base by Rebecca Jonas on 1/27/2020 6:27:26 PM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] high disk watermark [90%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%],, shards will be relocated away from this node

If the following warning is seen in the <APPIAN_HOME>\logs\search-server\search-server.log, see KB-1763 in addition to this article:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] flood stage disk watermark [95%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%], all indices on this node will be marked read-only

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

Reduce disk space on the server. KB-1255 provides guidance on how to address high disk usage of certain Appian components. If disk space cannot be reduced, add disk space. If the warning messages persist after disk has been reduced to a value below 90%, clear the search server indices. See the Action section of KB-1082 for steps on how to clear the search server indices.

The hardware requirements documented here provide general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: January 2020

Tags: administration, search server, disk usage, infrastructure, disk

KB-1766 "High disk watermark [90%] exceeded" warning seen in the search-server logs

Parmida Borhani — Fri, 06 Dec 2019 05:52:31 GMT

Revision 8 posted to Appian Knowledge Base by Parmida Borhani on 12/6/2019 5:52:31 AM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] high disk watermark [90%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%],, shards will be relocated away from this node

If the following warning is seen in the <APPIAN_HOME>\logs\search-server\search-server.log, see KB-1763 in addition to this article:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] flood stage disk watermark [95%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%], all indices on this node will be marked read-only

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

Reduce disk space on the server. KB-1255 provides guidance on how to address high disk usage of certain Appian components. If disk space cannot be reduced, add disk space. If the warning messages persist after disk has been reduced to a value below 90%, clear the search server indices. See the Action section of KB-1763 for steps on how to clear the search server indices.

The hardware requirements documented here provide general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: December 2019

Tags: administration, search server, disk usage, infrastructure, disk

KB-1766 "High disk watermark [90%] exceeded" warning seen in the search-server logs

Parmida Borhani — Tue, 19 Nov 2019 04:48:32 GMT

Revision 7 posted to Appian Knowledge Base by Parmida Borhani on 11/19/2019 4:48:32 AM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] high disk watermark [90%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%],, shards will be relocated away from this node

If the following warning is seen in the <APPIAN_HOME>\logs\search-server\search-server.log, see KB-1763 in addition to this article:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] flood stage disk watermark [95%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%], all indices on this node will be marked read-only

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

The hardware requirements documented here provide general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: November 2019

Tags: administration, search server, disk usage, infrastructure, disk

KB-1766 "High disk watermark [90%] exceeded" warning seen in the search-server logs

Parmida Borhani — Tue, 19 Nov 2019 04:47:41 GMT

Revision 6 posted to Appian Knowledge Base by Parmida Borhani on 11/19/2019 4:47:41 AM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] high disk watermark [90%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%],, shards will be relocated away from this node

If the following warning is seen in the <APPIAN_HOME>\logs\search-server\search-server.log, see KB-1763 in addition to this article:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node <HOSTNAME>:<SEARCH_SERVER_PORT>] flood stage disk watermark [95%] exceeded on [<HOSTNAME>][Node <HOSTNAME>:<SEARCH_SERVER_PORT>][<APPIAN_HOME>/search-server/data/nodes/0] free: X.Xgb[X.X%], all indices on this node will be marked read-only

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

The hardware requirements documented here provide general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: November 2019

Tags: administration, search server, infrastructure

KB-1766 "High disk watermark [90%] exceeded" warning seen in the search-server logs

Jordan Horwat — Wed, 31 Oct 2018 13:34:50 GMT

Revision 5 posted to Appian Knowledge Base by Jordan Horwat on 10/31/2018 1:34:50 PM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node *hostname*:9300] high disk watermark [90%] exceeded on [*hostname*][Node *hostname*:9300][<Appian_Home>\search-server\_admin\search-local\search-server\data\nodes\0] free: 23.2gb[9.7%], shards will be relocated away from this node

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

Add disk space and restart the search server. The hardware requirements documented here provide general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: October 2018

Tags: administration, search server, infrastructure

DRAFT KB-XXXX High disk watermark [90%] exceeded warning seen in the search-server logs

Jordan Horwat — Mon, 29 Oct 2018 16:35:27 GMT

Revision 4 posted to Appian Knowledge Base by Jordan Horwat on 10/29/2018 4:35:27 PM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node *hostname*:9300] high disk watermark [90%] exceeded on [*hostname*][Node *hostname*:9300][<Appian_Home>\search-server\_admin\search-local\search-server\data\nodes\0] free: 23.2gb[9.7%], shards will be relocated away from this node

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

Add disk space and restart the search server. The hardware requirements documented here provide general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: October 2018

Tags: administration, search server, infrastructure

DRAFT KB-XXXX High disk watermark [90%] exceeded warning seen in the search-server logs

Tejas Kargutkar — Mon, 29 Oct 2018 16:18:40 GMT

Revision 1 posted to Appian Knowledge Base by Tejas Kargutkar on 10/29/2018 4:18:40 PM

Symptoms

Warnings of the following type are observed in the <Appian_Home>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node *hostname*:9300] high disk watermark [90%] exceeded on [*hostname*][Node *hostname*:9300][<Appian_Home>\search-server\_admin\search-local\search-server\data\nodes\0] free: 23.2gb[9.7%], shards will be relocated away from this node

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

Add disk space and restart the search server. The hardware requirements documented here provides general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: October 2018

Tags: administration, search server, infrastructure

DRAFT KB-XXXX High disk watermark [90%] exceeded warning seen in the search-server logs

Tejas Kargutkar — Mon, 29 Oct 2018 12:22:05 GMT

Revision 3 posted to Appian Knowledge Base by Tejas Kargutkar on 10/29/2018 12:22:05 PM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node *hostname*:9300] high disk watermark [90%] exceeded on [*hostname*][Node *hostname*:9300][<Appian_Home>\search-server\_admin\search-local\search-server\data\nodes\0] free: 23.2gb[9.7%], shards will be relocated away from this node

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

Add disk space and restart the search server. The hardware requirements documented here provides general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: October 2018

Tags: administration, search server, infrastructure

DRAFT KB-XXXX High disk watermark [90%] exceeded warning seen in the search-server logs

Tejas Kargutkar — Mon, 29 Oct 2018 12:19:09 GMT

Revision 2 posted to Appian Knowledge Base by Tejas Kargutkar on 10/29/2018 12:19:09 PM

Symptoms

Warnings of the following type are observed in the <APPIAN_HOME>\logs\search-server\search-server.log:

[WARN ][org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor] [Node *hostname*:9300] high disk watermark [90%] exceeded on [*hostname*][Node *hostname*:9300][<Appian_Home>\search-server\_admin\search-local\search-server\data\nodes\0] free: 23.2gb[9.7%], shards will be relocated away from this node

Cause

This warning is seen when the disk usage crosses 90% of the available space on the server which hosts the search server.

Action

Add disk space and restart the search server. The hardware requirements documented here provides general guidelines regarding the necessary hardware allotment for an Appian environment.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: October 2018

Tags: administration, search server, infrastructure