[jira] [Commented] (MAPREDUCE-5157) Sort in hadoop-1 examples is not binary compatible with hadoop-2 mapred.lib

["Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13652137#comment-13652137 ] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-5157:
----------------------------------------------------

Looked at the patch. Some comments:
 - Did you try to implement the added-back Sampler's class's getSample using the super class implementation?
 - So all the added back code is just copy-pasted from branch-1? It did looks like so, I cross-verfied, except the class definitions which implement from mapreduce.* counterparts. Just confirming. Right?
 - The tests also seem to be strictly following their mapreduce counterparts, which is good except for one thing: There seems to be some difference in SplitSampler between 1.0 and the new one
    -- in 1.* we are using a SPLIT_STEP so as to only inspect by jumping around splits by that amount but read at a max that many splits as specified by the maxSplitsSampled parameter
    -- in 2.*, we are sampling the first maxSplitsSampled splits.
  This seems to be caused by MAPREDUCE-1820. Jumping around is in general preferable than the first N splits. Can you file a ticket?
                
> Sort in hadoop-1 examples is not binary compatible with hadoop-2 mapred.lib
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5157
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5157
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: MAPREDUCE-5157.1.patch
>
>
> In SORT example, org.apache.hadoop.mapred.lib.InputSampler.Sampler is used in hadoop-1. However, after upgrading to hadoop-2, org.apache.hadoop.mapred.lib.InputSampler is modified to extend org.apache.hadoop.mapreduce.lib.partition.InputSampler, and the inner class, Sampler, has been moved to the superclass. Therefore, hadoop-1 SORT can not find Sampler when it runs with hadoop-2 framework.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira