KB-2271 Number of terms exceeded the allowed maximum of designer-objects-ia in Search Server

Symptoms

Customers that have a large number of objects might see the following error in search-server.log in <APPIAN_HOME>/logs/search-server/.

[YYYY-MM-DDTHH:MM:SS.sssZ][DEBUG][org.elasticsearch.action.search.TransportSearchAction] [%node_name] All shards failed for phase: [query]
org.elasticsearch.index.query.QueryShardException: failed to create query: The number of terms [xxxxxxx] used in the Terms Query request has exceeded the allowed maximum of [65536]. This maximum can be set by changing the [index.max_terms_count] index level setting.
at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:369) ~[elasticsearch-7.9.1.jar:7.9.1]
...
Caused by: java.lang.IllegalArgumentException: The number of terms [xxxxxxx] used in the Terms Query request has exceeded the allowed maximum of [65536]. This maximum can be set by changing the [index.max_terms_count] index level setting.

During an attempt to open an application, the following system error is displayed on the front end:

Associated symptoms include but are not limited to:

Unable to look for dependents and precedents for certain objects.
Unable to look up certain objects with Quick Search Option.
Unable to open certain applications
Potential increase in Search Server CPU usage

Cause

The number of terms used in a term query has exceeded the allowed maximum of designer-objects-ia index in the Search Server, which is defined by index.max_terms_count. By default, the value of index.max_terms_count is 65536. See the Elasticsearch documentation for more information.

In Appian, the number of dependents/precedents that an object has equated to the number of terms. This does not necessarily mean that the customer has an object that has over 65536 dependents/precedents. The likelihood of reaching the term limit increases as the number of design objects and their dependencies grows.

Action

Follow the steps below to update the index.max_terms_count setting for the designer objects index. This operation will not cause downtime nor require Appian components to be restarted.

Note: For HA sites, the steps only need to be run on one of the nodes as the setting will be automatically applied to the other nodes. This setting will not be overridden on Search Server or App Server restarts.

Authenticate with Search Server by running the following command on the servers. Replace <APPIAN_HOME> with the absolute path of your own <APPIAN_HOME>:

APIKEY=$(awk '/^conf.data.search-server.restclient.apiKey=/ { match($0, /conf.data.search-server.restclient.apiKey=(.*)/, arr); print arr[1] }' <APPIAN_HOME>/conf/custom.properties); AUTHHEADER="Authorization: ApiKey $(echo -n $APIKEY | base64 -w0)"; alias curl='curl --header "$AUTHHEADER"'

Confirm the current value of index.max_terms_count (65536 is the default value):

curl 'localhost:9200/designer-objects-ia/_settings?pretty'| grep max_terms_count

If the output is expected, proceed to run the following commands to update the setting to your desired value, which should be higher than the value [xxxxxxx] shown in the error. Using 100000 as an example:
```
curl -X PUT localhost:9200/designer-objects-ia/_settings?pretty -H 'Content-Type: application/json' -d'
{
 "index" : {
 "max_terms_count" : "100000" 
 }
}
'
```
The output will show {"acknowledged" : true} if it succeeded.
Verify that the setting change took effect by running the following command. The setting should have the new value:
```
curl 'localhost:9200/designer-objects-ia/_settings?pretty'| grep max_terms_count
```
Note: If the site is HA, check on multiple nodes to be safe, be sure to run the authentication steps in step #1 on the other nodes first.
Once the steps are completed, please retry the erroring action that was throwing this error to confirm if it is resolved.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: November 2023