You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by arunmahadevan <gi...@git.apache.org> on 2018/09/14 18:41:57 UTC

[GitHub] storm pull request #2835: STORM-3222: Fix KafkaSpout internals to use Linked...

GitHub user arunmahadevan opened a pull request:

    https://github.com/apache/storm/pull/2835

    STORM-3222: Fix KafkaSpout internals to use LinkedList instead of ArrayList

    KafkaSpout internally maintains a waitingToEmit list per topic partition and keeps removing the first item to emit during each nextTuple. The implementation uses an ArrayList which results in un-necessary traversal and copy for each tuple.
    
    Also I am not sure why the nextTuple only emits a single tuple wheres ideally it should emit whatever it can emit in a single nextTuple call which is more efficient. However the logic appears too complicated to refactor.
    
    https://github.com/apache/storm/pull/2829 for 1.x

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/arunmahadevan/storm STORM-3222-1.x

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/storm/pull/2835.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2835
    
----
commit 2e0a01e5e4e36169ef2a45bf5dcd72792231ee07
Author: Arun Mahadevan <ar...@...>
Date:   2018-09-12T22:36:24Z

    STORM-3222: Fix KafkaSpout internals to use LinkedList instead of ArrayList
    
    KafkaSpout internally maintains a waitingToEmit list per topic partition and keeps removing the first item to emit during each nextTuple. The implementation uses an ArrayList which results in un-necessary traversal and copy for each tuple.
    
    Also I am not sure why the nextTuple only emits a single tuple wheres ideally it should emit whatever it can emit in a single nextTuple call which is more efficient. However the logic appears too complicated to refactor.

----


---

[GitHub] storm pull request #2835: STORM-3222: Fix KafkaSpout internals to use Linked...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/storm/pull/2835


---