You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/07/06 17:19:11 UTC
[jira] [Commented] (STORM-1946) ShellBolt.java - On busy system
BoltHeartbeatTimerTask fires before setHeartbeat() is executed
[ https://issues.apache.org/jira/browse/STORM-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364673#comment-15364673 ]
ASF GitHub Bot commented on STORM-1946:
---------------------------------------
GitHub user slava92 opened a pull request:
https://github.com/apache/storm/pull/1542
STORM-1946: initialize lastHeartbeatTimestamp before starting heartbeat task
When storm stars a large number of ShellBolt-s that consume a lot of CPU time to initialize, it creates a lot of contention between processes for CPU resource. That leads to BoltHeartbeatTimerTask being fired up after 1 second delay before setHeartbeat() assigns initial value to lastHeartbeatTimestamp variable.
As a result when BoltHeartbeatTimeTask fires up for the first time, getLastHeartbeat() returns value of 0. This in turn leads bolt to die with "subprocess heartbeat timeout" message.
The fix is to place setHeartBeat() before BoltHeartbeatTimerTask is created. The patch for this is attached.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/slava92/storm master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1542.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1542
----
commit d38c40026ab5dac266986896424e6c25a440e512
Author: Slava Andreyev <sa...@ets.org>
Date: 2016-07-06T17:14:09Z
initialize lastHeartbeatTimestamp before starting heartbeat task
----
> ShellBolt.java - On busy system BoltHeartbeatTimerTask fires before setHeartbeat() is executed
> ----------------------------------------------------------------------------------------------
>
> Key: STORM-1946
> URL: https://issues.apache.org/jira/browse/STORM-1946
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Reporter: Slava Andreyev
> Labels: patch
> Attachments: ShellBolt.java.patch
>
>
> When storm stars a large number of ShellBolt-s that consume a lot of CPU time to initialize, it creates a lot of contention between processes for CPU resource. That leads to [BoltHeartbeatTimerTask|https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/task/ShellBolt.java#L142] being fired up after 1 second delay _before_ [setHeartbeat()|https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/task/ShellBolt.java#L145] assigns initial value to [lastHeartbeatTimestamp|https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/task/ShellBolt.java#L91] variable.
> As a result when {{BoltHeartbeatTimeTask}} fires up for the first time, [getLastHeartbeat()|https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/task/ShellBolt.java#L316] returns value of *0*. This in turn leads bolt to die with ["subprocess heartbeat timeout"|https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/task/ShellBolt.java#L322] message.
> The fix is to place {{setHeartBeat()}} _before_ {{BoltHeartbeatTimerTask}} is created. The patch for this is attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)