[jira] Commented: (HADOOP-904) OutputFormat should be given the reduce id directly rather than a filename

["Doug Cutting (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465850 ] 

Doug Cutting commented on HADOOP-904:
-------------------------------------

+1 This looks like a good direction.

Passing "context" objects permits us to add and remove parameters to core interfaces without breaking implementations.  Instead of adding new parameters, breaking all implementations of the interface, we can instead add a new method to the context and deprecate old context methods.  We should move towards this style in more of the core mapreduce interfaces.

Also, I like keeping Path out of the core interface.  If we keep the mapreduce kernel free of FileSystem dependencies it makes it possible for folks to use mapreduce on non-FileSystem data, and also makes it easier for us to evolve the FileSystem apis, since their use in mapreduce will be localized to a few base classes.

> OutputFormat should be given the reduce id directly rather than a filename
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-904
>                 URL: https://issues.apache.org/jira/browse/HADOOP-904
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.10.1
>            Reporter: Owen O'Malley
>         Assigned To: Owen O'Malley
>             Fix For: 0.11.0
>
>
> The OutputFormat API should be changed to be more evolution proof:
> public interface OutputFormatContext {
>   JobConf getJobConf();
>   Progressable getProgress();
> }
> public interface OutputFormat {
>   RecordReader getRecordReader(int reduce, OutputFormatContext context) throws IOException;
>   void checkOutputSpecs(OutputFormatContext context) throws IOException;
> }
> And OutputFormatBase would be renamed:
> public abstract class FileOutputFormat implements OutputFormat {
>   protected Path getOutputPath(int reduce, OutputFormatContext context) throws IOException { ... }
>   ... current OutputFormatBase methods ...
> }

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira