You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mrunit.apache.org by "Jim Donofrio (Created) (JIRA)" <ji...@apache.org> on 2012/04/17 07:05:23 UTC
[jira] [Created] (MRUNIT-101) Offer the option to use real
OutputFormats instead of the MockOutputFormat
Offer the option to use real OutputFormats instead of the MockOutputFormat
--------------------------------------------------------------------------
Key: MRUNIT-101
URL: https://issues.apache.org/jira/browse/MRUNIT-101
Project: MRUnit
Issue Type: New Feature
Affects Versions: 0.9.0
Reporter: Jim Donofrio
Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves. This will require a good bit of worth as we need to write some kind of MemoryFileSystem class extending FileSystem since all of the OutputFormats want to write to Paths
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MRUNIT-101) Offer the option to use real
OutputFormats instead of the MockOutputFormat
Posted by "Jim Donofrio (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Donofrio updated MRUNIT-101:
--------------------------------
Fix Version/s: 1.0.0
> Offer the option to use real OutputFormats instead of the MockOutputFormat
> --------------------------------------------------------------------------
>
> Key: MRUNIT-101
> URL: https://issues.apache.org/jira/browse/MRUNIT-101
> Project: MRUnit
> Issue Type: New Feature
> Affects Versions: 0.9.0
> Reporter: Jim Donofrio
> Assignee: Jim Donofrio
> Labels: output_format
> Fix For: 1.0.0
>
>
> Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MRUNIT-101) Offer the option to use real
OutputFormats instead of the MockOutputFormat
Posted by "Jim Donofrio (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Donofrio updated MRUNIT-101:
--------------------------------
Description: Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves. (was: Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves. This will require a good bit of worth as we need to write some kind of MemoryFileSystem class extending FileSystem since all of the OutputFormats want to write to Paths)
> Offer the option to use real OutputFormats instead of the MockOutputFormat
> --------------------------------------------------------------------------
>
> Key: MRUNIT-101
> URL: https://issues.apache.org/jira/browse/MRUNIT-101
> Project: MRUnit
> Issue Type: New Feature
> Affects Versions: 0.9.0
> Reporter: Jim Donofrio
> Labels: output_format
>
> Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-101) allow use of real OutputFormats
such as TextOutputFormat instead of the
MockOutputCollector/MockContextWrapper classes
Posted by "Jim Donofrio (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272173#comment-13272173 ]
Jim Donofrio commented on MRUNIT-101:
-------------------------------------
initial checkin in 1336519, still need to cleanup the javadoc
> allow use of real OutputFormats such as TextOutputFormat instead of the MockOutputCollector/MockContextWrapper classes
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MRUNIT-101
> URL: https://issues.apache.org/jira/browse/MRUNIT-101
> Project: MRUnit
> Issue Type: New Feature
> Affects Versions: 0.9.0
> Reporter: Jim Donofrio
> Assignee: Jim Donofrio
> Labels: output_format
> Fix For: 1.0.0
>
>
> Currently we only use the MockOutputCollector/MockContextWrapper which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MRUNIT-101) Offer the option to use real
OutputFormats instead of the MockOutputFormat
Posted by "Jim Donofrio (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Donofrio reassigned MRUNIT-101:
-----------------------------------
Assignee: Jim Donofrio
> Offer the option to use real OutputFormats instead of the MockOutputFormat
> --------------------------------------------------------------------------
>
> Key: MRUNIT-101
> URL: https://issues.apache.org/jira/browse/MRUNIT-101
> Project: MRUnit
> Issue Type: New Feature
> Affects Versions: 0.9.0
> Reporter: Jim Donofrio
> Assignee: Jim Donofrio
> Labels: output_format
>
> Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-101) Offer the option to use real
OutputFormats instead of the MockOutputFormat
Posted by "Jim Donofrio (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257471#comment-13257471 ]
Jim Donofrio commented on MRUNIT-101:
-------------------------------------
I will add a method withOutputFormat(Class<? extends FileOutputFormat> outputFormat, Class<? extends FileInputFormat> inputFormat). I think we should move to users passing in classes to be more consistent with the conf instead of instances of outputformats, mappers, etc. We can only really test classes that extend FileOutputFormat because there is no contract on how to add input/output to the other types. I will create a random dir mrunit-random number in java.io.tmpdir. I will add this path to the FileOutputFormat. I will use the FileOutputFormat to get a record writer and write out the output using a MockOutputCollector. Then using the provided FileInputFormat I will add the output file and call getSplits and then getRecordReader which will allow me to read the output back in and compare the results to the expected output using the traditional validate method
> Offer the option to use real OutputFormats instead of the MockOutputFormat
> --------------------------------------------------------------------------
>
> Key: MRUNIT-101
> URL: https://issues.apache.org/jira/browse/MRUNIT-101
> Project: MRUnit
> Issue Type: New Feature
> Affects Versions: 0.9.0
> Reporter: Jim Donofrio
> Labels: output_format
>
> Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MRUNIT-101) allow use of real OutputFormats such
as TextOutputFormat instead of the MockOutputCollector/MockContextWrapper
classes
Posted by "Jim Donofrio (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Donofrio updated MRUNIT-101:
--------------------------------
Description: Currently we only use the MockOutputCollector/MockContextWrapper which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves. (was: Currently we only use the MockOutputFormats which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves.)
Summary: allow use of real OutputFormats such as TextOutputFormat instead of the MockOutputCollector/MockContextWrapper classes (was: Offer the option to use real OutputFormats instead of the MockOutputFormat)
> allow use of real OutputFormats such as TextOutputFormat instead of the MockOutputCollector/MockContextWrapper classes
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MRUNIT-101
> URL: https://issues.apache.org/jira/browse/MRUNIT-101
> Project: MRUnit
> Issue Type: New Feature
> Affects Versions: 0.9.0
> Reporter: Jim Donofrio
> Assignee: Jim Donofrio
> Labels: output_format
> Fix For: 1.0.0
>
>
> Currently we only use the MockOutputCollector/MockContextWrapper which just copy the output key, value into a list. This prevents users from testing the outputformat for classes such as TextOutputFormat which should compare the toString versions of objects for equality not the objects themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira