You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mrunit.apache.org by Thamizhannal <an...@gmail.com> on 2012/10/10 13:53:49 UTC
MRunit test program issue
Hi,
I am experimenting MRUnit. I have a MR program written in hadoop-0.19 and
Below is my mrunit test case program.
Though my expected & actual outpupts got matched, the test case got failed.
my configs:
hadoop-0.19
mrunit-0.9.0-incubating-hadoop1.jar
junit-4.8.2
mockito-all-1.8.5
Please expalin me what am I missing.
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mrunit.MapDriver;
import org.apache.hadoop.mrunit.MapReduceDriver;
import org.apache.hadoop.mrunit.ReduceDriver;
import org.junit.Before;
import org.junit.Test;
public class LogParseMRUnitTest {
MapDriver<LongWritable, Text, Text, Text> mapDriver;
String line1 = ""; // some text given here
Log log1 = new Log();
@Before
public void setup() {
LogParseMapper mapper = new LogParseMapper();
mapDriver = new MapDriver<LongWritable, Text, Text, Text();
mapDriver.setMapper(mapper);
Configuration conf = new Configuration();
// add config here as needed
mapDriver.setConfiguration(conf);
}
@Test
public void testMapper() {
mapDriver.withInput(new LongWritable(), new Text(line1));
log1.populateLogData(line1.trim().toLowerCase());
String[] fields = line1.split(" ");
String timeKey = null;
if (fields.length >= 1) {
cal.setTimeInMillis(Long.parseLong(fields[0]) * 1000);
timeKey = sdf.format(cal.getTime());
}
mapDriver.withOutput(new Text(timeKey ), log1);
mapDriver.runTest();
}
}
output:
12/10/10 16:58:50 ERROR mrunit.TestDriver: Missing expected output
(2012081710, {text}) at position 0.
12/10/10 16:58:50 ERROR mrunit.TestDriver: Received unexpected output
(2012081710,{text}) at position 0.
Thanks
Thamizh
Re: MRunit test program issue
Posted by Joey Echeverria <jo...@cloudera.com>.
FYI, hadoop-0.19 is a very old release. I'd recommend upgrading to
hadoop 1.x or taking a look at a Hadoop distribution.
-Joey
On Wed, Oct 10, 2012 at 8:22 AM, Thamizhannal <an...@gmail.com> wrote:
> Thanks you so much Dave. It worked me.
>
>
> On Wed, Oct 10, 2012 at 5:33 PM, Dave Beech <da...@paraliatech.com> wrote:
>>
>> Hi Thamizh
>>
>> The problem will be in your Log class. Have you overrridden equals and
>> hashCode?
>>
>> You should do this in order to provide a true comparison of your objects.
>> Otherwise you're simply comparing the object references, which will not
>> match
>>
>> Thanks
>> Dave
>>
>> On 10 Oct 2012, at 12:53, Thamizhannal <an...@gmail.com> wrote:
>>
>> > Hi,
>> > I am experimenting MRUnit. I have a MR program written in hadoop-0.19
>> > and Below is my mrunit test case program.
>> > Though my expected & actual outpupts got matched, the test case got
>> > failed.
>> > my configs:
>> > hadoop-0.19
>> > mrunit-0.9.0-incubating-hadoop1.jar
>> > junit-4.8.2
>> > mockito-all-1.8.5
>> >
>> > Please expalin me what am I missing.
>> >
>> > import org.apache.hadoop.conf.Configuration;
>> > import org.apache.hadoop.io.LongWritable;
>> > import org.apache.hadoop.io.Text;
>> > import org.apache.hadoop.mrunit.MapDriver;
>> > import org.apache.hadoop.mrunit.MapReduceDriver;
>> > import org.apache.hadoop.mrunit.ReduceDriver;
>> > import org.junit.Before;
>> > import org.junit.Test;
>> >
>> >
>> > public class LogParseMRUnitTest {
>> > MapDriver<LongWritable, Text, Text, Text> mapDriver;
>> >
>> > String line1 = ""; // some text given here
>> >
>> > Log log1 = new Log();
>> >
>> > @Before
>> > public void setup() {
>> > LogParseMapper mapper = new LogParseMapper();
>> > mapDriver = new MapDriver<LongWritable, Text, Text, Text();
>> > mapDriver.setMapper(mapper);
>> >
>> > Configuration conf = new Configuration();
>> > // add config here as needed
>> > mapDriver.setConfiguration(conf);
>> > }
>> >
>> > @Test
>> > public void testMapper() {
>> > mapDriver.withInput(new LongWritable(), new Text(line1));
>> >
>> > log1.populateLogData(line1.trim().toLowerCase());
>> > String[] fields = line1.split(" ");
>> > String timeKey = null;
>> > if (fields.length >= 1) {
>> > cal.setTimeInMillis(Long.parseLong(fields[0]) * 1000);
>> > timeKey = sdf.format(cal.getTime());
>> > }
>> >
>> > mapDriver.withOutput(new Text(timeKey ), log1);
>> >
>> > mapDriver.runTest();
>> > }
>> > }
>> > output:
>> > 12/10/10 16:58:50 ERROR mrunit.TestDriver: Missing expected output
>> > (2012081710, {text}) at position 0.
>> > 12/10/10 16:58:50 ERROR mrunit.TestDriver: Received unexpected output
>> > (2012081710,{text}) at position 0.
>> > Thanks
>> > Thamizh
>
>
>
>
> --
> Regards,
> Thamizhannal
--
Joey Echeverria
Principal Solutions Architect
Cloudera, Inc.
Re: MRunit test program issue
Posted by Thamizhannal <an...@gmail.com>.
Thanks you so much Dave. It worked me.
On Wed, Oct 10, 2012 at 5:33 PM, Dave Beech <da...@paraliatech.com> wrote:
> Hi Thamizh
>
> The problem will be in your Log class. Have you overrridden equals and
> hashCode?
>
> You should do this in order to provide a true comparison of your objects.
> Otherwise you're simply comparing the object references, which will not
> match
>
> Thanks
> Dave
>
> On 10 Oct 2012, at 12:53, Thamizhannal <an...@gmail.com> wrote:
>
> > Hi,
> > I am experimenting MRUnit. I have a MR program written in hadoop-0.19
> and Below is my mrunit test case program.
> > Though my expected & actual outpupts got matched, the test case got
> failed.
> > my configs:
> > hadoop-0.19
> > mrunit-0.9.0-incubating-hadoop1.jar
> > junit-4.8.2
> > mockito-all-1.8.5
> >
> > Please expalin me what am I missing.
> >
> > import org.apache.hadoop.conf.Configuration;
> > import org.apache.hadoop.io.LongWritable;
> > import org.apache.hadoop.io.Text;
> > import org.apache.hadoop.mrunit.MapDriver;
> > import org.apache.hadoop.mrunit.MapReduceDriver;
> > import org.apache.hadoop.mrunit.ReduceDriver;
> > import org.junit.Before;
> > import org.junit.Test;
> >
> >
> > public class LogParseMRUnitTest {
> > MapDriver<LongWritable, Text, Text, Text> mapDriver;
> >
> > String line1 = ""; // some text given here
> >
> > Log log1 = new Log();
> >
> > @Before
> > public void setup() {
> > LogParseMapper mapper = new LogParseMapper();
> > mapDriver = new MapDriver<LongWritable, Text, Text, Text();
> > mapDriver.setMapper(mapper);
> >
> > Configuration conf = new Configuration();
> > // add config here as needed
> > mapDriver.setConfiguration(conf);
> > }
> >
> > @Test
> > public void testMapper() {
> > mapDriver.withInput(new LongWritable(), new Text(line1));
> >
> > log1.populateLogData(line1.trim().toLowerCase());
> > String[] fields = line1.split(" ");
> > String timeKey = null;
> > if (fields.length >= 1) {
> > cal.setTimeInMillis(Long.parseLong(fields[0]) * 1000);
> > timeKey = sdf.format(cal.getTime());
> > }
> >
> > mapDriver.withOutput(new Text(timeKey ), log1);
> >
> > mapDriver.runTest();
> > }
> > }
> > output:
> > 12/10/10 16:58:50 ERROR mrunit.TestDriver: Missing expected output
> (2012081710, {text}) at position 0.
> > 12/10/10 16:58:50 ERROR mrunit.TestDriver: Received unexpected output
> (2012081710,{text}) at position 0.
> > Thanks
> > Thamizh
>
--
Regards,
Thamizhannal
Re: MRunit test program issue
Posted by Dave Beech <da...@paraliatech.com>.
Hi Thamizh
The problem will be in your Log class. Have you overrridden equals and hashCode?
You should do this in order to provide a true comparison of your objects. Otherwise you're simply comparing the object references, which will not match
Thanks
Dave
On 10 Oct 2012, at 12:53, Thamizhannal <an...@gmail.com> wrote:
> Hi,
> I am experimenting MRUnit. I have a MR program written in hadoop-0.19 and Below is my mrunit test case program.
> Though my expected & actual outpupts got matched, the test case got failed.
> my configs:
> hadoop-0.19
> mrunit-0.9.0-incubating-hadoop1.jar
> junit-4.8.2
> mockito-all-1.8.5
>
> Please expalin me what am I missing.
>
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.io.LongWritable;
> import org.apache.hadoop.io.Text;
> import org.apache.hadoop.mrunit.MapDriver;
> import org.apache.hadoop.mrunit.MapReduceDriver;
> import org.apache.hadoop.mrunit.ReduceDriver;
> import org.junit.Before;
> import org.junit.Test;
>
>
> public class LogParseMRUnitTest {
> MapDriver<LongWritable, Text, Text, Text> mapDriver;
>
> String line1 = ""; // some text given here
>
> Log log1 = new Log();
>
> @Before
> public void setup() {
> LogParseMapper mapper = new LogParseMapper();
> mapDriver = new MapDriver<LongWritable, Text, Text, Text();
> mapDriver.setMapper(mapper);
>
> Configuration conf = new Configuration();
> // add config here as needed
> mapDriver.setConfiguration(conf);
> }
>
> @Test
> public void testMapper() {
> mapDriver.withInput(new LongWritable(), new Text(line1));
>
> log1.populateLogData(line1.trim().toLowerCase());
> String[] fields = line1.split(" ");
> String timeKey = null;
> if (fields.length >= 1) {
> cal.setTimeInMillis(Long.parseLong(fields[0]) * 1000);
> timeKey = sdf.format(cal.getTime());
> }
>
> mapDriver.withOutput(new Text(timeKey ), log1);
>
> mapDriver.runTest();
> }
> }
> output:
> 12/10/10 16:58:50 ERROR mrunit.TestDriver: Missing expected output (2012081710, {text}) at position 0.
> 12/10/10 16:58:50 ERROR mrunit.TestDriver: Received unexpected output (2012081710,{text}) at position 0.
> Thanks
> Thamizh