You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2011/05/17 22:29:47 UTC
[jira] [Created] (MAPREDUCE-2506) Create a compatible interface for
frameworks that need to clone MapReduce context objects.
Create a compatible interface for frameworks that need to clone MapReduce context objects.
------------------------------------------------------------------------------------------
Key: MAPREDUCE-2506
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2506
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Owen O'Malley
In 0.21 we moved the org.apache.hadoop.mapreduce context objects to interfaces.
That made the APIs much better, but broke backwards compatibility for frameworks that need to clone them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2506) Create a compatible interface for
frameworks that need to clone MapReduce context objects.
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated MAPREDUCE-2506:
-------------------------------------
Attachment: (was: ContextFactory.java)
> Create a compatible interface for frameworks that need to clone MapReduce context objects.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2506
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2506
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Owen O'Malley
> Attachments: ContextFactory.java, mr-2506.patch
>
>
> In 0.21 we moved the org.apache.hadoop.mapreduce context objects to interfaces.
> That made the APIs much better, but broke backwards compatibility for frameworks that need to clone them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2506) Create a compatible interface for
frameworks that need to clone MapReduce context objects.
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated MAPREDUCE-2506:
-------------------------------------
Comment: was deleted
(was: This version of it compiles and runs against unmodified versions of 0.20.2, 0.20.203.0, and 0.21.0.)
> Create a compatible interface for frameworks that need to clone MapReduce context objects.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2506
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2506
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Owen O'Malley
> Attachments: ContextFactory.java, mr-2506.patch
>
>
> In 0.21 we moved the org.apache.hadoop.mapreduce context objects to interfaces.
> That made the APIs much better, but broke backwards compatibility for frameworks that need to clone them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2506) Create a compatible interface
for frameworks that need to clone MapReduce context objects.
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035065#comment-13035065 ]
Owen O'Malley commented on MAPREDUCE-2506:
------------------------------------------
This patch creates a ContextFactory class that provides two functions:
{code}
public static JobContext cloneContext(JobContext original,
Configuration conf
) throws IOException,
InterruptedException;
public static <K1,V1,K2,V2> Mapper<K1,V1,K2,V2>.Context
cloneMapContext(MapContext<K1,V1,K2,V2> context,
Configuration conf,
RecordReader<K1,V1> reader,
RecordWriter<K2,V2> writer
) throws IOException, InterruptedException;
{code}
This first clones JobContexts, TaskAttemptContexts, and MapContexts. The contexts are the same, except they have a new configuration.
The second method allows additional hooks to replace the reader and writer for the context.
> Create a compatible interface for frameworks that need to clone MapReduce context objects.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2506
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2506
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Owen O'Malley
> Attachments: ContextFactory.java, ContextFactory.java, mr-2506.patch
>
>
> In 0.21 we moved the org.apache.hadoop.mapreduce context objects to interfaces.
> That made the APIs much better, but broke backwards compatibility for frameworks that need to clone them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2506) Create a compatible interface for
frameworks that need to clone MapReduce context objects.
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated MAPREDUCE-2506:
-------------------------------------
Attachment: ContextFactory.java
This version of it compiles and runs against unmodified versions of 0.20.2, 0.20.203.0, and 0.21.0.
> Create a compatible interface for frameworks that need to clone MapReduce context objects.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2506
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2506
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Owen O'Malley
> Attachments: ContextFactory.java, ContextFactory.java, mr-2506.patch
>
>
> In 0.21 we moved the org.apache.hadoop.mapreduce context objects to interfaces.
> That made the APIs much better, but broke backwards compatibility for frameworks that need to clone them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2506) Create a compatible interface for
frameworks that need to clone MapReduce context objects.
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated MAPREDUCE-2506:
-------------------------------------
Attachment: ContextFactory.java
This patch I wrote for Pig that compiles and runs against unmodified 0.20.2, 0.20.203.0, and 0.21.0.
> Create a compatible interface for frameworks that need to clone MapReduce context objects.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2506
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2506
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Owen O'Malley
> Attachments: ContextFactory.java, mr-2506.patch
>
>
> In 0.21 we moved the org.apache.hadoop.mapreduce context objects to interfaces.
> That made the APIs much better, but broke backwards compatibility for frameworks that need to clone them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2506) Create a compatible interface for
frameworks that need to clone MapReduce context objects.
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated MAPREDUCE-2506:
-------------------------------------
Attachment: mr-2506.patch
> Create a compatible interface for frameworks that need to clone MapReduce context objects.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2506
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2506
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Owen O'Malley
> Attachments: mr-2506.patch
>
>
> In 0.21 we moved the org.apache.hadoop.mapreduce context objects to interfaces.
> That made the APIs much better, but broke backwards compatibility for frameworks that need to clone them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira