You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Bikas Saha (JIRA)" <ji...@apache.org> on 2014/03/20 03:05:44 UTC

[jira] [Commented] (TEZ-961) Incorrect split generation

    [ https://issues.apache.org/jira/browse/TEZ-961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941311#comment-13941311 ] 

Bikas Saha commented on TEZ-961:
--------------------------------

The total length reported is actually 224 and not 0. The underlying InputFormat generated 4 splits. 

If this repros easily then debug logs could be turned on to show why its creating 3 groups instead of 1.

> Incorrect split generation
> --------------------------
>
>                 Key: TEZ-961
>                 URL: https://issues.apache.org/jira/browse/TEZ-961
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>            Reporter: Vikram Dixit K
>
> 2014-03-19 17:55:29,044 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.split.TezMapredSplitsGrouper: Desired splits: 1 too large.  Desired splitLength: 224 Min splitLength: 16777216 New desired splits: 1 Total length: 224 Original splits: 4
> The split is actually 0 sized but we are creating 4 initial splits and 3 grouped splits (all of size 0).



--
This message was sent by Atlassian JIRA
(v6.2#6252)