[jira] [Updated] (MAPREDUCE-5974) Allow map output collector fallback

["Todd Lipcon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated MAPREDUCE-5974:
-----------------------------------

    Attachment: mapreduce-5974.txt

Attached patch implements the improvement as described. I did not include any new unit test since this code path is exercised by existing paths, and mocking out this section of the code is really quite difficult. If folks would like to see a unit test, I can add a full functional test which specifies some bogus collector class ahead of the real implementation, but figured that it's a trivial enough change we could get by without.

> Allow map output collector fallback
> -----------------------------------
>
>                 Key: MAPREDUCE-5974
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5974
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: task
>    Affects Versions: 2.6.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: mapreduce-5974.txt
>
>
> Currently we only allow specifying a single MapOutputCollector implementation class in a job. It would be nice to allow a comma-separated list of classes: we should try each collector implementation in the user-specified order until we find one that can be successfully instantiated and initted.
> This is useful for cases where a particular optimized collector implementation cannot operate on all key/value types, or requires native code. The cluster administrator can configure the cluster to try to use the optimized collector and fall back to the default collector.



--
This message was sent by Atlassian JIRA
(v6.2#6252)