[jira] [Commented] (ARTEMIS-1078) Thread pool executor occasionally misses to create a new thread

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARTEMIS-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945290#comment-15945290 ] 

ASF GitHub Bot commented on ARTEMIS-1078:
-----------------------------------------

GitHub user bgutjahr opened a pull request:

    https://github.com/apache/activemq-artemis/pull/1144

    ARTEMIS-1078: Improved thread pool

    The improved thread pool make sure that only those threads that are currently waiting in queue poll or take calls are considered idle and ready to run a queued task. If not enough threads are idle, new threads will be created - up to the maximum number.
    
    The case where this affected is was in HA backup failover, when client reconnect tasks went into an endless loop due to an unlimited number of reconnect attempts, preventing the task to handle the failover from being executed due to the missed thread creation. This fix will help as long as the thread pool's maximum number of threads has not been reached, or some of the currently running threads will not hang in such endless loops and can run the failover task without significant delay. I would suggest to recheck the looping reconnect code. Maybe it can be rewritten to submit a new task to try the next reconnect, such that other queued tasks could also be given a chance to be executed.
    
    I created the fix on the 1.x branch. I guess this will also affect the master branch.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/bgutjahr/activemq-artemis improved-thread-pool

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/activemq-artemis/pull/1144.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1144
    
----
commit 07edd3968f668aacf44f8f984a2c35d98552af99
Author: Bernd Gutjahr <be...@hpe.com>
Date:   2017-03-28T13:54:58Z

    ARTEMIS-1078 Improved thread pool to consider only threads waiting for the queue to get new tasks as idle.
    
    The thread pool maintained a counter of active threads, but that counter was increased too late in the beforeExecute method. Submitting a task created a new thread. If now a second task was submitter before the new thread had started to execute it's task, the second task was queued without creating a 2nd thread. So the second task was only executed after the first task had been completed - even if the thread pool's maximum number of thread had not been reached.
    
    This fix now only count those threads that are currently waiting in the queue's poll or take methods as idle threads, and creates new threads unless there are enough idle threads to pick up all queued tasks.
    
    	modified:   artemis-commons/src/main/java/org/apache/activemq/artemis/utils/ActiveMQThreadPoolExecutor.java

----


> Thread pool executor occasionally misses to create a new thread
> ---------------------------------------------------------------
>
>                 Key: ARTEMIS-1078
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-1078
>             Project: ActiveMQ Artemis
>          Issue Type: Bug
>    Affects Versions: 1.5.2
>            Reporter: Bernd Gutjahr
>
> The thread pool executor class ActiveMQThreadPoolExecutor occasionally misses to create a new thread. This happens if a thread for another task has just been created but not yet started to work on its task. If that first task runs for a long time, the new task will be deferred until the first task has completed.
> In our case, the occasionally blocked HA backup failover, when the failover-handler was not run because an infinetely looping client connection reconnect task blocked the thread. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)