[jira] [Resolved] (AMBARI-14671) Reevaluate the use of threadpools in Ambari code base

["Jayush Luniya (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayush Luniya resolved AMBARI-14671.
------------------------------------
    Resolution: Invalid

This is already addressed.

> Reevaluate the use of threadpools in Ambari code base
> -----------------------------------------------------
>
>                 Key: AMBARI-14671
>                 URL: https://issues.apache.org/jira/browse/AMBARI-14671
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.2.0
>            Reporter: Jayush Luniya
>            Assignee: Jayush Luniya
>             Fix For: 3.0.0
>
>         Attachments: DynamicScaling.tgz
>
>
> As part of investigation for BUG-43981, noticed that in many places in the Ambari code base, the way we use the threadpool is not quite correct.  We will never scale up the number of threads on high load when we use unbounded queues. This could lead to performance bottlenecks especially if we are configure ThreadPoolExecutor with corePoolSize=0 and maxPoolSize=10, only one thread will ever be spawned. See observations below
> Observations: 
> 1. When a ThreadPoolExecutor object is created, the pool size is 0 (i.e. no new threads are created then) unless prestartAllCoreThreads() is called. Also if we set allowCoreThreadTimeOut(true), idle core threads will also be reclaimed.
> 2. In our code base, I observed that we create a threadpool using unlimited queue. However the pool will never scale up from coreThreads -> maxThreads as the request will always get queued. 
> {code:java}
> LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>(); // unlimited Queue
> ThreadPoolExecutor threadPoolExecutor =
>     new ThreadPoolExecutor(
>         THREAD_POOL_CORE_SIZE,
>         THREAD_POOL_MAX_SIZE,
>         THREAD_POOL_TIMEOUT_MILLIS,
>         TimeUnit.MILLISECONDS,
>         queue);
> {code}
> http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
> {quote}
> Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without a predefined capacity) will cause new tasks to wait in the queue when all corePoolSize threads are busy. Thus, no more than corePoolSize threads will ever be created. (And the value of the maximumPoolSize therefore doesn't have any effect.) This may be appropriate when each task is completely independent of others, so tasks cannot affect each others execution; for example, in a web page server. While this style of queuing can be useful in smoothing out transient bursts of requests, it admits the possibility of unbounded work queue growth when commands continue to arrive on average faster than they can be processed.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)