You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "Xingyu Su (JIRA)" <ji...@apache.org> on 2015/07/09 03:07:04 UTC
[jira] [Updated] (STORM-929) High CPU usage when bolt idle due to
short disruptor queue wait time
[ https://issues.apache.org/jira/browse/STORM-929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xingyu Su updated STORM-929:
----------------------------
Description:
I'm running topology which has large num of executors (500) on storm 0.9.3. I find the CPU usage over 100% when topology idle. And half of the CPU usage is from kernel. I look into CPU utilization of worker process and find most of threads stop at:
com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit)
I use Storm starter topology (wordcounter) to reproduce this issue. I change the sleep time of spout to 10s and executor num of bolt to 500. So there was effectively no task to do. Again the CPU usage comes to 100% and half from kernel. I think this may caused by frequently switching thread context due to short disruptor queue wait time.
> High CPU usage when bolt idle due to short disruptor queue wait time
> --------------------------------------------------------------------
>
> Key: STORM-929
> URL: https://issues.apache.org/jira/browse/STORM-929
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.9.3
> Reporter: Xingyu Su
>
> I'm running topology which has large num of executors (500) on storm 0.9.3. I find the CPU usage over 100% when topology idle. And half of the CPU usage is from kernel. I look into CPU utilization of worker process and find most of threads stop at:
> com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit)
> I use Storm starter topology (wordcounter) to reproduce this issue. I change the sleep time of spout to 10s and executor num of bolt to 500. So there was effectively no task to do. Again the CPU usage comes to 100% and half from kernel. I think this may caused by frequently switching thread context due to short disruptor queue wait time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)