You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2010/05/17 00:33:43 UTC

[jira] Resolved: (CASSANDRA-1096) Sequential splits causing unbalanced MapReduce load

     [ https://issues.apache.org/jira/browse/CASSANDRA-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis resolved CASSANDRA-1096.
---------------------------------------

      Assignee: Joost Ouwerkerk
    Resolution: Fixed

committed.  nice troubleshooting work!

> Sequential splits causing unbalanced MapReduce load
> ---------------------------------------------------
>
>                 Key: CASSANDRA-1096
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1096
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Hadoop
>    Affects Versions: 0.6
>            Reporter: Joost Ouwerkerk
>            Assignee: Joost Ouwerkerk
>            Priority: Minor
>             Fix For: 0.6.2
>
>         Attachments: CASSANDRA-1096.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Since CassandraInputFormat returns an ordered list of splits, when there are many splits (e.g. hundreds or more) the load on cassandra is horribly unbalanced.  e.g. if I have 30 tasks processing 600 splits, then the rows for the first 30 splits are all located on the same one or two nodes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.