You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2013/08/23 02:15:52 UTC

[jira] [Created] (MAPREDUCE-5478) TeraInputFormat unnecessarily defines its own FileSplit subclass

Sandy Ryza created MAPREDUCE-5478:
-------------------------------------

             Summary: TeraInputFormat unnecessarily defines its own FileSplit subclass
                 Key: MAPREDUCE-5478
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5478
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: examples
    Affects Versions: 2.1.0-beta
            Reporter: Sandy Ryza
            Assignee: Sandy Ryza
            Priority: Minor


TeraInputFormat defines its own TeraFileSplit subclass of FileSplit that adds a locations field, which is already included in FileSplit.

This is causing MR2 TeraSort to fail on MR1, which, for a System.arraycopy, requires splits to be of the FileSplit class.  While nobody is promising that everything that runs on MR2 should run on MR1, fixing this would be easy and make it possible to compare MR2 TeraSort performance between MR1 and MR2.

We should just get rid of TeraFileSplit and use FileSplit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira