You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/10/29 21:32:27 UTC
[jira] [Commented] (STORM-1145) Have the IConnection push batches
instead of buffering them
[ https://issues.apache.org/jira/browse/STORM-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981206#comment-14981206 ]
ASF GitHub Bot commented on STORM-1145:
---------------------------------------
GitHub user revans2 opened a pull request:
https://github.com/apache/storm/pull/830
STORM-1145: Have IConnection push tuples instead of pull them.
This pull request is based off of #765 because it moved the overflow buffer into the disruptor queue itself, and we don't want the netty thread to block when doing routing.
This does not improve the maximum throughput much, but because it removes an extra thread and an extra queue that a tuple must go through when being sent to another worker the 99%-ile, 99.9%-ile and worst case latency goes down some.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/revans2/incubator-storm server-callback
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/830.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #830
----
commit c415d321cc7cb5e5a7bf91ab9a309e9504896fbb
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Date: 2015-09-22T20:37:25Z
Added batching of tuples at a disruptor queue level.
commit a7165d7f5a291502aa0fa4fd77ac0b19dcc3eca7
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Date: 2015-10-28T21:18:08Z
STORM-1145: Have IConnection push tuples instead of pull them.
Move deserialization to the IConnectionCallback so only Tuple batches are flowing through the system, not unserialized byte arrays.
----
> Have the IConnection push batches instead of buffering them
> -----------------------------------------------------------
>
> Key: STORM-1145
> URL: https://issues.apache.org/jira/browse/STORM-1145
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
>
> The messaging layer currently buffers tuples and waits for one or more threads to take the tuples and route them where they need to go.
> This adds an extra thread that a tuple has to go through before it can be processed, and places the tuple in a LinkedBlockingQueue waiting for one of the threads to be ready to process the data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)