You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mrunit.apache.org by Thamizhannal <an...@gmail.com> on 2012/10/19 13:35:48 UTC

HBase Mapper MRUnit Test case issue

Hi,

I have Mapper only Job that uploads data to Hbase and I have written MRunit
test case for that.
Though, my expected and actual outputs are matching(Text match) MRunit
throws below error message
12/10/19 16:38:16 ERROR mrunit.TestDriver: Missing expected output ()
12/10/19 16:38:16 ERROR mrunit.TestDriver: Received unexpected output ()

My config files:
mrunit-0.9.0-incubating-hadoop1.jar
mockito-all-1.8.5.jar
junit-4.10.jar

Test Case code:
---------------
    public void testMapper() throws Exception {
        Text key = new Text("txt here");
        Text value = new Text("txt here");
        mapDriver.withInput(new Text(key), new Text(value));
        Map<ImmutableBytesWritable, Put> map = computeHbaseUpdateMap(key,
value);
        for (Map.Entry<ImmutableBytesWritable, Put> entry : map.entrySet())
{
            mapDriver.addOutput(entry.getKey(), entry.getValue());

            System.out.println(entry.getKey() + "," + entry.getValue());
        }
        mapDriver.runTest(false);
    }

The exception happens on validate() of TestDriver.java. Here
expectedPositions and expectedPositions maps have same content.
Here, since output has been derived from expectedPositions in the source
code. The expectedPositionList takes [0] and actualPositionList becomes
null.
When I replace expectedPositions with actualPositions on for loop the
result becomes vice versa.

    final Map<Pair<K2, V2>, List<Integer>> expectedPositions =
buildPositionMap(expectedOutputs);
    final Map<Pair<K2, V2>, List<Integer>> actualPositions =
buildPositionMap(outputs);
    for (final Pair<K2, V2> output : expectedPositions.keySet()) {
        final List<Integer> expectedPositionList =
expectedPositions.get(output);
        final List<Integer> actualPositionList =
actualPositions.get(output);
        -----------------------------
        ----------------------
    }
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.mrunit/mrunit/0.9.0-incubating/org/apache/hadoop/mrunit/TestDriver.java

As per my code Pari<K2,V2> = Map<ImmutableBytesWritable, Put>. Is it
problem with ImmutableBytesWritable Or Put implementation ?

Any help would be appreciated.

Thanks,
Tamil

Re: HBase Mapper MRUnit Test case issue

Posted by Brock Noland <br...@cloudera.com>.
Hi,

Would you be willing to share you entire unit test so I can run it myself?

Brock

On Fri, Oct 19, 2012 at 6:35 AM, Thamizhannal <an...@gmail.com> wrote:

> Hi,
>
> I have Mapper only Job that uploads data to Hbase and I have written
> MRunit test case for that.
> Though, my expected and actual outputs are matching(Text match) MRunit
> throws below error message
> 12/10/19 16:38:16 ERROR mrunit.TestDriver: Missing expected output ()
> 12/10/19 16:38:16 ERROR mrunit.TestDriver: Received unexpected output ()
>
> My config files:
> mrunit-0.9.0-incubating-hadoop1.jar
> mockito-all-1.8.5.jar
> junit-4.10.jar
>
> Test Case code:
> ---------------
>     public void testMapper() throws Exception {
>         Text key = new Text("txt here");
>         Text value = new Text("txt here");
>         mapDriver.withInput(new Text(key), new Text(value));
>         Map<ImmutableBytesWritable, Put> map = computeHbaseUpdateMap(key,
> value);
>         for (Map.Entry<ImmutableBytesWritable, Put> entry :
> map.entrySet()) {
>             mapDriver.addOutput(entry.getKey(), entry.getValue());
>
>             System.out.println(entry.getKey() + "," + entry.getValue());
>         }
>         mapDriver.runTest(false);
>     }
>
> The exception happens on validate() of TestDriver.java. Here
> expectedPositions and expectedPositions maps have same content.
> Here, since output has been derived from expectedPositions in the source
> code. The expectedPositionList takes [0] and actualPositionList becomes
> null.
> When I replace expectedPositions with actualPositions on for loop the
> result becomes vice versa.
>
>     final Map<Pair<K2, V2>, List<Integer>> expectedPositions =
> buildPositionMap(expectedOutputs);
>     final Map<Pair<K2, V2>, List<Integer>> actualPositions =
> buildPositionMap(outputs);
>     for (final Pair<K2, V2> output : expectedPositions.keySet()) {
>         final List<Integer> expectedPositionList =
> expectedPositions.get(output);
>         final List<Integer> actualPositionList =
> actualPositions.get(output);
>         -----------------------------
>         ----------------------
>     }
>
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.mrunit/mrunit/0.9.0-incubating/org/apache/hadoop/mrunit/TestDriver.java
>
> As per my code Pari<K2,V2> = Map<ImmutableBytesWritable, Put>. Is it
> problem with ImmutableBytesWritable Or Put implementation ?
>
> Any help would be appreciated.
>
> Thanks,
> Tamil
>



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/