You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@storm.apache.org by "Jungtaek Lim (JIRA)" <ji...@apache.org> on 2017/08/04 04:20:00 UTC

[jira] [Created] (STORM-2678) Improve performance of LoadAwareShuffleGrouping

Jungtaek Lim created STORM-2678:
-----------------------------------

             Summary: Improve performance of LoadAwareShuffleGrouping 
                 Key: STORM-2678
                 URL: https://issues.apache.org/jira/browse/STORM-2678
             Project: Apache Storm
          Issue Type: Bug
          Components: storm-client
            Reporter: Jungtaek Lim
            Assignee: Jungtaek Lim


While reviewing STORM-2306, I found a room to improve LoadAwareShuffleGrouping, basically taking similar approach with ShuffleGrouping.

The idea is constructing virtual ring (array) with proper length which tasks are randomly distributed to the ring respecting their loads, and just do sequential access like we do in ShuffleGrouping. Maybe slower to reflect load information due to construct virtual ring but no need to call randInt and additional loop for each tuple emission.

We may also want to reduce call of System.currentTimeMillis(), and if we get rid of that, it should be better.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)