IO task worker threads increase rapidly on JBoss EAP 7.0.x and 7.1.x

Solution Verified - Updated

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP) 7.x

Issue

  • There are 0 IO worker task threads at startup, but these rapidly grow to our task-max-threads setting even with a light load on JBoss EAP 7.0.x and 7.1.x.
  • And it looks the behavior has changed on JBoss EAP 7.2.x or later. Is there any detailed information about this change?

Resolution

JBoss EAP 7.0/7.1

It's expected behavior as our task worker thread pool implementation (XNIO's Content from github.com is not included.WorkerThread$TaskPool) is based on Java's Content from docs.oracle.com is not included.ThreadPoolExecutor on JBoss EAP 7.0.x/7.1.x:

When a new task is submitted in method execute(java.lang.Runnable), and fewer than corePoolSize threads are running, a new thread is created to handle the request, even if other worker threads are idle

The IO task pool uses an executor with corePoolSize equal to maximumPoolSize, where size is calculated by 16 * number of CPU cores ​by default. Since core=max in this pool, this means each new task processed by the thread pool creates a new thread in the pool until you reach your configured thread pool max. But once you reach the max, thread creation ceases and threads are indefinitely reused without concern.

​JBoss EAP 7.2 or later

A new worker thread pool implementation has been incorporated in JBoss EAP 7.2 through This content is not included.WFLY-5332 / This content is not included.WFCORE-3397 / This content is not included.JBTHR-38. This new worker thread pool implementation is Content from github.com is not included.JBoss Threads' EnhancedQueueExecutor that extends Java's Content from docs.oracle.com is not included.AbstractExecutorService and has the following desirable characteristics:

  • Idle threads are always reused before queueing tasks or starting new threads. The most recently used idle thread is always preferred.
  • The number of task-core-threads (corePoolSize) and task-max-threads (maximumPoolSize) can be configurable independently. Hence, task-keepalive can shrink the thread pool to core-threads when it's idle.

See also Content from lists.jboss.org is not included.this mail thread on wildfly-dev for the details about the new worker thread pool implementation.

SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.