KB-1159 "Could not obtain 3 thread(s) after X attempts in work poller" error printed in application server log

Symptoms

The following error is displayed in the application server log:

[Appian WorkPoller - 0 [Notifications] (originally: default-threads - 19)] ERROR com.appiancorp.ra.workpoller.WorkPoller - Could not obtain 3 thread(s) after X attempts in work poller

Cause

The application server allocates up to 30 workpoller threads to receive and process work from the Appian engines. This issue occurs when all 30 workpoller threads are in use, preventing the engines from sending new work to the application server.

Action

To resolve the issue, perform the following:

  1. Identify the top threads by CPU.
    1. In Linux, run top, press the c key, then press Shift+H. This will show the threads sorted by highest to lowest CPU usage. Take a screenshot of the output.

    2. In Windows, use Process Explorer or Process Monitor to show the threads for the Java process and take a screenshot of the top threads by CPU.

  2. Take multiple thread dumps during the time of the issue
    1. Determine the process id of the application server process:
      For JBoss: ps -ef | grep standalone
      For Tomcat: ps -ef | grep bootstrap
    2. Take the thread dump:
      jstack processid > /path/threaddump-YYYYMMDD-HHMM.txt
    3. Repeat Step 2 every minute for 5 minutes.
  3. Restart the application server
  4. Analyze the thread dumps to determine the cause of the long running threads.

The usual culprits are:

  • Looping functions
  • Unresponsive or slow external server (database, mail server, or other integration)
  • Custom plugin that contains a long running call

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: November 2019

Related
Recommended