PipelineMapReduceDriver w/ 'mapreduce' api, and testing multiple K,V inputs

[David Parks <da...@yahoo.com>]

2 questions:

 

1)      PipelineMapReduceDriver hasn’t been upgraded to support the
org.apache.hadoop.mapreduce.Mapper/Reducer api has it? Or is that in the
works?

 

2)      Using just the MapReduceDriver or the MapDriver or ReduceDriver
alone, the tests only run a single key/value through the class, then the
next input fully re-creates the class right? If I’m not wrong in that
statement then this seems like something that can be improved. I just noted
a case where we incorrectly kept state between two different key/value
entries in a map task and the issue could only be reproduced in the cluster.
A better test, it seems, would allow multiple input values and iterate
through them as hadoop really does.  Am I on track here?

 

Thanks,

Dave

 

 

From: 周锋 [mailto:zhoufenglokki@gmail.com] 
Sent: Friday, February 01, 2013 10:02 AM
To: user@mrunit.apache.org
Subject: can't attach mrunit sources jar with "mvn -DdownloadSources=true
eclipse:eclipse"

 

can't attach sources jar with "mvn -U -DdownloadSources=true
-DdownloadJavadocs=false eclipse:clean eclipse:eclipse"

 

need help, thanks!


my dependency is with classifier:
        <dependency>
            <groupId>org.apache.mrunit</groupId>
            <artifactId>mrunit</artifactId>
            <version>0.9.0-incubating</version>
            <classifier>hadoop1</classifier>
            <scope>test</scope>
        </dependency>

my maven-eclipse-plugin:

        <plugin>
            <artifactId>maven-eclipse-plugin</artifactId>
            <version>2.9</version>
        </plugin>

 

my maven 3.0.4 uses maven central repository:
http://search.maven.org/#browse|1513871525

 

my workaround:

download "mrunit-0.9.0-incubating-sources.jar" with classifier "sources",
then rename to "mrunit-0.9.0-incubating-hadoop1-sources.jar"

 

lokki
2013-2-1