[jira] Created: (MAPREDUCE-2002) MRUnit driver classes should provide ability to set a configuration object to be passed into the mapper/reducer

["David Rosenstrauch (JIRA)" <ji...@apache.org>]

MRUnit driver classes should provide ability to set a configuration object to be passed into the mapper/reducer
---------------------------------------------------------------------------------------------------------------

                 Key: MAPREDUCE-2002
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2002
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: contrib/mrunit
    Affects Versions: 0.20.2
            Reporter: David Rosenstrauch
            Priority: Minor


Short description:

Enhance the org.apache.hadoop.mrunit.mapreduce.MapDriver, ReduceDriver, and MapReduceDriver unit test driver classes to contain "setConfiguration" and "withConfiguration" methods for passing in user-supplied org.apache.hadoop.conf.Configuration objects, and have those configuration objects eventually get passed on to the Context objects that are passed in to the mapper/reducer "setup" methods.  (Rather than passing in an empty Configuration object, as is being done now.)


Long description:

The MRUnit driver classes (i.e., MapDriver, ReduceDriver, and MapReduceDriver) ought to be enhanced to contain methods for setting a Configuration object to be used by the mapper/reducer being tested - i.e., setConfiguration() and withConfiguration().

The only way to effectively pass parameters into a mapper or reducer is by setting properties on a configuration object, which the mapper/reducer can then retrieve in their "setup" step, and use to customize its operation.  As a result, specific mappers/reducers may require the presence of specific configuration properties/parameters in order to function correctly (or at all).  (I am currently coding such a reducer right now.)

Testing such a mapper/reducer thus requires that the unit testing framework used provide the ability to pass in user-supplied Configuration objects to them so that they can be tested with appropriate parameter values.  However, MRUnit currently does not provide this ability.  (All mappers/reducers are always passed an empty configuration object.)  And there is not even currently any (easy) way for the end-user to fix this problem by creating a simple sub-class that supplies this functionality, as such subclasses would require a substantial reimplementation/override of several MRUnit framework classes.

I believe this something that is not too difficult to fix in the MRUnit framework code, however, and would greatly help the usability of MRUnit.


Although I don't have time to code this enhancement right now, if needed/preferred I could squeeze out some time to code up a patch for this.  If that's needed, please let me know.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.