You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Mathias Herberts (Created) (JIRA)" <ji...@apache.org> on 2011/10/29 19:23:32 UTC

[jira] [Created] (MAPREDUCE-3310) Custom grouping comparator cannot be set for Combiners

Custom grouping comparator cannot be set for Combiners
------------------------------------------------------

                 Key: MAPREDUCE-3310
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3310
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: client
    Affects Versions: 0.21.0, 0.20.205.0, 0.20.203.0, 0.20.2, 0.20.1
         Environment: All
            Reporter: Mathias Herberts


Combiners are often described as 'Reducers running on the Map side'.

As Reducers, Combiners are fed <K,{V}>, where {V} is built by grouping values associated with the 'same' key.

For Reducers, the comparator used for grouping values can be set independently of that used to sort the keys (using Job.setGroupingComparatorClass).

Such a configuration is not possible for Combiners, meaning some things done in Reducers cannot be done in Combiners (such as secondary sort).

It would be handy to have a Job.setCombinerGroupingComparatorClass method that would allow the setting of the grouping comparator used when applying a Combiner.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira