[jira] [Commented] (MAPREDUCE-3710) last split generated by FileInputFormat.getSplits may not have the best locality

["Hadoop QA (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191427#comment-13191427 ] 

Hadoop QA commented on MAPREDUCE-3710:
--------------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12511536/MR-3710_v1.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 6 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in .

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1652//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1652//console

This message is automatically generated.
                
> last split generated by FileInputFormat.getSplits may not have the best locality
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3710
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3710
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv1, mrv2
>    Affects Versions: 0.23.0, 1.0.0
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>         Attachments: MR-3710_v1.txt
>
>
> The last split generated by FileInputFormat.getSplits considers {{blkLocations.length-1}} to be the hosts for the split.
> The last split may be larger than the rest (SPLIT_SLOP=1.1 by default) - in which case locality is picked up from a smaller block.
> e.g. 1027MB file with a 128MB split size. The last split ends up being 131MB. The hosts for locality end up being the nodes containing the 3MB block instead of the 128MB block.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira