You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mrunit.apache.org by Thamizhannal <an...@gmail.com> on 2012/10/19 13:35:48 UTC
HBase Mapper MRUnit Test case issue
Hi,
I have Mapper only Job that uploads data to Hbase and I have written MRunit
test case for that.
Though, my expected and actual outputs are matching(Text match) MRunit
throws below error message
12/10/19 16:38:16 ERROR mrunit.TestDriver: Missing expected output ()
12/10/19 16:38:16 ERROR mrunit.TestDriver: Received unexpected output ()
My config files:
mrunit-0.9.0-incubating-hadoop1.jar
mockito-all-1.8.5.jar
junit-4.10.jar
Test Case code:
---------------
public void testMapper() throws Exception {
Text key = new Text("txt here");
Text value = new Text("txt here");
mapDriver.withInput(new Text(key), new Text(value));
Map<ImmutableBytesWritable, Put> map = computeHbaseUpdateMap(key,
value);
for (Map.Entry<ImmutableBytesWritable, Put> entry : map.entrySet())
{
mapDriver.addOutput(entry.getKey(), entry.getValue());
System.out.println(entry.getKey() + "," + entry.getValue());
}
mapDriver.runTest(false);
}
The exception happens on validate() of TestDriver.java. Here
expectedPositions and expectedPositions maps have same content.
Here, since output has been derived from expectedPositions in the source
code. The expectedPositionList takes [0] and actualPositionList becomes
null.
When I replace expectedPositions with actualPositions on for loop the
result becomes vice versa.
final Map<Pair<K2, V2>, List<Integer>> expectedPositions =
buildPositionMap(expectedOutputs);
final Map<Pair<K2, V2>, List<Integer>> actualPositions =
buildPositionMap(outputs);
for (final Pair<K2, V2> output : expectedPositions.keySet()) {
final List<Integer> expectedPositionList =
expectedPositions.get(output);
final List<Integer> actualPositionList =
actualPositions.get(output);
-----------------------------
----------------------
}
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.mrunit/mrunit/0.9.0-incubating/org/apache/hadoop/mrunit/TestDriver.java
As per my code Pari<K2,V2> = Map<ImmutableBytesWritable, Put>. Is it
problem with ImmutableBytesWritable Or Put implementation ?
Any help would be appreciated.
Thanks,
Tamil
Re: HBase Mapper MRUnit Test case issue
Posted by Brock Noland <br...@cloudera.com>.
Hi,
Would you be willing to share you entire unit test so I can run it myself?
Brock
On Fri, Oct 19, 2012 at 6:35 AM, Thamizhannal <an...@gmail.com> wrote:
> Hi,
>
> I have Mapper only Job that uploads data to Hbase and I have written
> MRunit test case for that.
> Though, my expected and actual outputs are matching(Text match) MRunit
> throws below error message
> 12/10/19 16:38:16 ERROR mrunit.TestDriver: Missing expected output ()
> 12/10/19 16:38:16 ERROR mrunit.TestDriver: Received unexpected output ()
>
> My config files:
> mrunit-0.9.0-incubating-hadoop1.jar
> mockito-all-1.8.5.jar
> junit-4.10.jar
>
> Test Case code:
> ---------------
> public void testMapper() throws Exception {
> Text key = new Text("txt here");
> Text value = new Text("txt here");
> mapDriver.withInput(new Text(key), new Text(value));
> Map<ImmutableBytesWritable, Put> map = computeHbaseUpdateMap(key,
> value);
> for (Map.Entry<ImmutableBytesWritable, Put> entry :
> map.entrySet()) {
> mapDriver.addOutput(entry.getKey(), entry.getValue());
>
> System.out.println(entry.getKey() + "," + entry.getValue());
> }
> mapDriver.runTest(false);
> }
>
> The exception happens on validate() of TestDriver.java. Here
> expectedPositions and expectedPositions maps have same content.
> Here, since output has been derived from expectedPositions in the source
> code. The expectedPositionList takes [0] and actualPositionList becomes
> null.
> When I replace expectedPositions with actualPositions on for loop the
> result becomes vice versa.
>
> final Map<Pair<K2, V2>, List<Integer>> expectedPositions =
> buildPositionMap(expectedOutputs);
> final Map<Pair<K2, V2>, List<Integer>> actualPositions =
> buildPositionMap(outputs);
> for (final Pair<K2, V2> output : expectedPositions.keySet()) {
> final List<Integer> expectedPositionList =
> expectedPositions.get(output);
> final List<Integer> actualPositionList =
> actualPositions.get(output);
> -----------------------------
> ----------------------
> }
>
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.mrunit/mrunit/0.9.0-incubating/org/apache/hadoop/mrunit/TestDriver.java
>
> As per my code Pari<K2,V2> = Map<ImmutableBytesWritable, Put>. Is it
> problem with ImmutableBytesWritable Or Put implementation ?
>
> Any help would be appreciated.
>
> Thanks,
> Tamil
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/