You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mrunit.apache.org by "Jim Donofrio (Commented) (JIRA)" <ji...@apache.org> on 2012/03/11 00:08:57 UTC
[jira] [Commented] (MRUNIT-13) Add support for MultipleOutputs
[ https://issues.apache.org/jira/browse/MRUNIT-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226955#comment-13226955 ]
Jim Donofrio commented on MRUNIT-13:
------------------------------------
To do this I guess we need to have a MockRecordWriter returned from a MockOutputFormat. Users will specify everything that they usually specify for MultipleOutputs except for the OutputFormat. The MockRecordWriter will store all the results in a list as we do now with the MockOutputCollector. Then the problem is how we pass this list back to the MapDriver or ReduceDriver. The only shared object we would have access to is the JobConf. We can then use the Serialization classes to serialize the objects out to bytes, convert the bytes into a string using their hex representation to prevent the String decode from messing anything up. Then we can store the hex strings in the conf and deserialize them out in the MapDriver or ReduceDriver to verify the actual output matches the requested output for each multipleoutputs type.
I recognize this isnt the cleanest solution but this will be easier for the user and I am not sure this is another way of doing this. Any thoughts?
> Add support for MultipleOutputs
> -------------------------------
>
> Key: MRUNIT-13
> URL: https://issues.apache.org/jira/browse/MRUNIT-13
> Project: MRUnit
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: E. Sammer
> Assignee: Jim Donofrio
> Fix For: 1.0.0
>
>
> Add support to mrunit for Hadoop's MultipleOutputs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira