Comparing classes vs. instanceof in TestDriver

[Scott Eade <sc...@eade.id.au>]

Hi,

I don't want my test case to rely on the same processing as the job it is
testing.  To this end, rather than using the output class of the job I want
to extend the output class to include direct access to internals for
testing purposes.

I would like to have:

public class MyWritable implemnts Writable {
    protected int someValue = 0;

    public doSomething(int increment) { // invoked in reducer
        someValue += increment; // contrived example
    }
    // and a bunch of other stuff
}

public class MyWritableTestExt extends MyWritable {
    public setSomeValue(int someValue) { // invoked in test case
        this.someValue = someValue;
    }
}

My reducer can emit MyWritable after invoking doSomething() as necessary as
part of its processing.  In my test case I can expect a MyWritableTestExt
that I configure using setSomeValue().

Sounds good, but my test fails with:
Mismatch in value class: expected: MyWritableTestExt actual: MyWritable

Would it make sense for
org.apache.hadoop.mrunit.TestDriver.checkTypesAndLogError() to use:
    if (expectedValueClass instanceof actualValueClass)
rahter than
    if (actualValueClass != expectedValueClass)
to check the types?

The same might apply to keys as well as values.

Is this a sensible suggestion or am I missing something?  It seems that
without this either my test case has to replicate the reducer code (not
ideal) or I have to pollute the MyWritable with test code.

Thanks,

Scott

Re: Comparing classes vs. instanceof in TestDriver

[Bertrand Dechoux <de...@gmail.com>]

>From what I understand, MRUnit might indeed be wrong and what you are
asking could be justifiable but I am not 100% sure.
Anyway, the way to go is to write a unit test for MRUnit before modifying
anything. Would you be able to provide it?

Here is my understand of your context.
* R<?,V1,?,V1> is your reducer class
* V1 extends V2 and you would like to be able to use V2 with R knowing that
there is no impact to the R code
* MRUnit rely on reflection to find what the class of value is and assume
the class found should be expected one

For production code, it is the user which is responsible for setting
explicitly which class should be used. The demand is sensible it is only
that MRUnit magic configuration isn't smart enough.

Another way to fix that would be to rely on the classes configured in the
Configuration. It seems more bothersome for the user but it would guarantee
that test cases do not mix different subclasses of the defined parent class.

Bertrand


On Thu, Jul 11, 2013 at 6:40 AM, Scott Eade <sc...@eade.id.au> wrote:

> Hi,
>
> I don't want my test case to rely on the same processing as the job it is
> testing.  To this end, rather than using the output class of the job I want
> to extend the output class to include direct access to internals for
> testing purposes.
>
> I would like to have:
>
> public class MyWritable implemnts Writable {
>     protected int someValue = 0;
>
>     public doSomething(int increment) { // invoked in reducer
>         someValue += increment; // contrived example
>     }
>     // and a bunch of other stuff
> }
>
> public class MyWritableTestExt extends MyWritable {
>     public setSomeValue(int someValue) { // invoked in test case
>         this.someValue = someValue;
>     }
> }
>
> My reducer can emit MyWritable after invoking doSomething() as necessary
> as part of its processing.  In my test case I can expect a
> MyWritableTestExt that I configure using setSomeValue().
>
> Sounds good, but my test fails with:
> Mismatch in value class: expected: MyWritableTestExt actual: MyWritable
>
> Would it make sense for
> org.apache.hadoop.mrunit.TestDriver.checkTypesAndLogError() to use:
>     if (expectedValueClass instanceof actualValueClass)
> rahter than
>     if (actualValueClass != expectedValueClass)
> to check the types?
>
> The same might apply to keys as well as values.
>
> Is this a sensible suggestion or am I missing something?  It seems that
> without this either my test case has to replicate the reducer code (not
> ideal) or I have to pollute the MyWritable with test code.
>
> Thanks,
>
> Scott
>