You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Srimanth Gunturi <sg...@hortonworks.com> on 2016/01/14 19:01:04 UTC
ThreadPoolExecutor being used incorrectly
Hello,
Wanted to write this down and have discussion about 'java.util.concurrent.ThreadPoolExecutor' being used incorrectly in Ambari.
Almost all usages call the below constructor:
public ThreadPoolExecutor(
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue)
Where typical values used are:
new ThreadPoolExecutor(
20,
100,
30000L,
TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()) // Unbounded queue
The weird thing about 'ThreadPoolExecutor' is that the number of threads increases from the 'corePoolSize' (20) to the 'maximumPoolSize' (100) ONLY when the BlockingQueue is full. Whenever we use an unbounded blocking queue, the number of threads never goes past the 'corePoolSize'.
The javadoc states: "If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full"
So our thinking that the Executor will automatically increase the number of threads upto 'maximumPoolSize' and store overflowing requests into the queue (unbounded) is incorrect. Due to our usage, the number of threads never increases beyond 'corePoolSize'.
I do not understand the reason why it is implemented this way, but we run into performance issues by getting stuck with 'corePoolSize' thread count. I am looking into a fix for using ThreadPoolExecutor where the number of threads increases upto 'maximumPoolSize'.
Regards,
Srimanth?
Re: ThreadPoolExecutor being used incorrectly
Posted by "Harp, Michael" <Mi...@Teradata.com>.
Construct the LinkedBlockingQueue with a fixed capacity rather than the default, Integer.MAX_VALUE?
Mike
On 1/14/16, 10:01 AM, "Srimanth Gunturi" <sg...@hortonworks.com> wrote:
>Hello,
>Wanted to write this down and have discussion about 'java.util.concurrent.ThreadPoolExecutor' being used incorrectly in Ambari.
>
>Almost all usages call the below constructor:
>
>public ThreadPoolExecutor(
> int corePoolSize,
> int maximumPoolSize,
> long keepAliveTime,
> TimeUnit unit,
> BlockingQueue<Runnable> workQueue)
>
>Where typical values used are:
>
>new ThreadPoolExecutor(
> 20,
> 100,
> 30000L,
> TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue<Runnable>()) // Unbounded queue
>
>The weird thing about 'ThreadPoolExecutor' is that the number of threads increases from the 'corePoolSize' (20) to the 'maximumPoolSize' (100) ONLY when the BlockingQueue is full. Whenever we use an unbounded blocking queue, the number of threads never goes past the 'corePoolSize'.
>
>The javadoc states: "If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full"
>
>So our thinking that the Executor will automatically increase the number of threads upto 'maximumPoolSize' and store overflowing requests into the queue (unbounded) is incorrect. Due to our usage, the number of threads never increases beyond 'corePoolSize'.
>
>I do not understand the reason why it is implemented this way, but we run into performance issues by getting stuck with 'corePoolSize' thread count. I am looking into a fix for using ThreadPoolExecutor where the number of threads increases upto 'maximumPoolSize'.
>
>Regards,
>Srimanth?
>
Re: ThreadPoolExecutor being used incorrectly
Posted by Jayush Luniya <jl...@hortonworks.com>.
Sounds good Srimanth.
Thanks
Jayush
On 1/14/16, 10:40 AM, "Srimanth Gunturi" <sg...@hortonworks.com> wrote:
>Hi Jayush,
>It is very nice that you have noticed the same issue as well and have a
>solution - I will look into it.
>
>I have a working patch myself where I use a custom rejection handler that
>relies on a secondary queue (unbounded), and made the Executor's queue
>bounded (like Michael Harp mentioned).
>
>I will give both a spin and write down in JIRA.
>Regards,
>Srimanth
>
>
>
>________________________________________
>From: Jayush Luniya <jl...@hortonworks.com>
>Sent: Thursday, January 14, 2016 10:32 AM
>To: dev@ambari.apache.org
>Subject: Re: ThreadPoolExecutor being used incorrectly
>
>Hi Srimanth,
>
>I recently noticed this behavior too. I filed an external JIRA with a
>prototype to address this issue.
>https://issues.apache.org/jira/browse/AMBARI-14671
>
>
>Regards
>Jayush
>
>On 1/14/16, 10:01 AM, "Srimanth Gunturi" <sg...@hortonworks.com> wrote:
>
>>Hello,
>>Wanted to write this down and have discussion about
>>'java.util.concurrent.ThreadPoolExecutor' being used incorrectly in
>>Ambari.
>>
>>Almost all usages call the below constructor:
>>
>>public ThreadPoolExecutor(
>> int corePoolSize,
>> int maximumPoolSize,
>> long keepAliveTime,
>> TimeUnit unit,
>> BlockingQueue<Runnable> workQueue)
>>
>>Where typical values used are:
>>
>>new ThreadPoolExecutor(
>> 20,
>> 100,
>> 30000L,
>> TimeUnit.MILLISECONDS,
>> new LinkedBlockingQueue<Runnable>()) // Unbounded queue
>>
>>The weird thing about 'ThreadPoolExecutor' is that the number of threads
>>increases from the 'corePoolSize' (20) to the 'maximumPoolSize' (100)
>>ONLY when the BlockingQueue is full. Whenever we use an unbounded
>>blocking queue, the number of threads never goes past the 'corePoolSize'.
>>
>>The javadoc states: "If there are more than corePoolSize but less than
>>maximumPoolSize threads running, a new thread will be created only if the
>>queue is full"
>>
>>So our thinking that the Executor will automatically increase the number
>>of threads upto 'maximumPoolSize' and store overflowing requests into the
>>queue (unbounded) is incorrect. Due to our usage, the number of threads
>>never increases beyond 'corePoolSize'.
>>
>>I do not understand the reason why it is implemented this way, but we run
>>into performance issues by getting stuck with 'corePoolSize' thread
>>count. I am looking into a fix for using ThreadPoolExecutor where the
>>number of threads increases upto 'maximumPoolSize'.
>>
>>Regards,
>>Srimanth?
>>
>
>
>
Re: ThreadPoolExecutor being used incorrectly
Posted by Srimanth Gunturi <sg...@hortonworks.com>.
Hi Jayush,
It is very nice that you have noticed the same issue as well and have a solution - I will look into it.
I have a working patch myself where I use a custom rejection handler that relies on a secondary queue (unbounded), and made the Executor's queue bounded (like Michael Harp mentioned).
I will give both a spin and write down in JIRA.
Regards,
Srimanth
________________________________________
From: Jayush Luniya <jl...@hortonworks.com>
Sent: Thursday, January 14, 2016 10:32 AM
To: dev@ambari.apache.org
Subject: Re: ThreadPoolExecutor being used incorrectly
Hi Srimanth,
I recently noticed this behavior too. I filed an external JIRA with a
prototype to address this issue.
https://issues.apache.org/jira/browse/AMBARI-14671
Regards
Jayush
On 1/14/16, 10:01 AM, "Srimanth Gunturi" <sg...@hortonworks.com> wrote:
>Hello,
>Wanted to write this down and have discussion about
>'java.util.concurrent.ThreadPoolExecutor' being used incorrectly in
>Ambari.
>
>Almost all usages call the below constructor:
>
>public ThreadPoolExecutor(
> int corePoolSize,
> int maximumPoolSize,
> long keepAliveTime,
> TimeUnit unit,
> BlockingQueue<Runnable> workQueue)
>
>Where typical values used are:
>
>new ThreadPoolExecutor(
> 20,
> 100,
> 30000L,
> TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue<Runnable>()) // Unbounded queue
>
>The weird thing about 'ThreadPoolExecutor' is that the number of threads
>increases from the 'corePoolSize' (20) to the 'maximumPoolSize' (100)
>ONLY when the BlockingQueue is full. Whenever we use an unbounded
>blocking queue, the number of threads never goes past the 'corePoolSize'.
>
>The javadoc states: "If there are more than corePoolSize but less than
>maximumPoolSize threads running, a new thread will be created only if the
>queue is full"
>
>So our thinking that the Executor will automatically increase the number
>of threads upto 'maximumPoolSize' and store overflowing requests into the
>queue (unbounded) is incorrect. Due to our usage, the number of threads
>never increases beyond 'corePoolSize'.
>
>I do not understand the reason why it is implemented this way, but we run
>into performance issues by getting stuck with 'corePoolSize' thread
>count. I am looking into a fix for using ThreadPoolExecutor where the
>number of threads increases upto 'maximumPoolSize'.
>
>Regards,
>Srimanth?
>
Re: ThreadPoolExecutor being used incorrectly
Posted by Jayush Luniya <jl...@hortonworks.com>.
Hi Srimanth,
I recently noticed this behavior too. I filed an external JIRA with a
prototype to address this issue.
https://issues.apache.org/jira/browse/AMBARI-14671
Regards
Jayush
On 1/14/16, 10:01 AM, "Srimanth Gunturi" <sg...@hortonworks.com> wrote:
>Hello,
>Wanted to write this down and have discussion about
>'java.util.concurrent.ThreadPoolExecutor' being used incorrectly in
>Ambari.
>
>Almost all usages call the below constructor:
>
>public ThreadPoolExecutor(
> int corePoolSize,
> int maximumPoolSize,
> long keepAliveTime,
> TimeUnit unit,
> BlockingQueue<Runnable> workQueue)
>
>Where typical values used are:
>
>new ThreadPoolExecutor(
> 20,
> 100,
> 30000L,
> TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue<Runnable>()) // Unbounded queue
>
>The weird thing about 'ThreadPoolExecutor' is that the number of threads
>increases from the 'corePoolSize' (20) to the 'maximumPoolSize' (100)
>ONLY when the BlockingQueue is full. Whenever we use an unbounded
>blocking queue, the number of threads never goes past the 'corePoolSize'.
>
>The javadoc states: "If there are more than corePoolSize but less than
>maximumPoolSize threads running, a new thread will be created only if the
>queue is full"
>
>So our thinking that the Executor will automatically increase the number
>of threads upto 'maximumPoolSize' and store overflowing requests into the
>queue (unbounded) is incorrect. Due to our usage, the number of threads
>never increases beyond 'corePoolSize'.
>
>I do not understand the reason why it is implemented this way, but we run
>into performance issues by getting stuck with 'corePoolSize' thread
>count. I am looking into a fix for using ThreadPoolExecutor where the
>number of threads increases upto 'maximumPoolSize'.
>
>Regards,
>Srimanth?
>