You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mrunit.apache.org by "Jakub (JIRA)" <ji...@apache.org> on 2012/11/10 13:07:11 UTC
[jira] [Created] (MRUNIT-159) MRUnit not compiling with hadoop
0.23.3
Jakub created MRUNIT-159:
----------------------------
Summary: MRUnit not compiling with hadoop 0.23.3
Key: MRUNIT-159
URL: https://issues.apache.org/jira/browse/MRUNIT-159
Project: MRUnit
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Jakub
Priority: Critical
I tried to write test using mrunit 1.0.0 ( i need to init cache, which is unavaliable in already relased versions)
when I run my code I get:
java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
at org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon(AbstractMockContextWrapper.java:53)
I read other issues like https://issues.apache.org/jira/browse/MRUNIT-56 and decided to recompile mrunit.
I've downloaded source, and then:
mvn install -DskipTests
Build succes
mvn install -DskipTests -Dhadoop.version=2
Build success
tried both output version with my code:
mrunit-1.0.0-SNAPSHOT-hadoop1.jar
mrunit-1.0.0-SNAPSHOT-hadoop2.jar
but was still getting exception
I've changed pom.xml specificHadoopVersion to:
<specificHadoopVersion>0.23.3</specificHadoopVersion>
and then:
mvn clean
mvn install -DskipTests -Dhadoop.version=2
and there are compilation errors:
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[464,24] error: cannot find symbol
[ERROR] class DistributedCache
/home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[478,24] error: cannot find symbol
[INFO] 2 errors
that's because initDistributedCache method uses DistributedCache.addLocalFiles(), which is unavaliable in Hadoop 0.23.3
I tried to fix it, but I'm new to hadoop ant havent succeed yet, any help, please?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-159) MRUnit not compiling with hadoop
0.23.3
Posted by "Dave Beech (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494870#comment-13494870 ]
Dave Beech commented on MRUNIT-159:
-----------------------------------
Hi Jakub - thanks for reporting this. I've replaced the use of the missing addLocalFiles and addLocalArchives methods with our own helper methods which set the cache config parameters directly. Can you test this and let me know if it solves your issue?
> MRUnit not compiling with hadoop 0.23.3
> ---------------------------------------
>
> Key: MRUNIT-159
> URL: https://issues.apache.org/jira/browse/MRUNIT-159
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jakub
> Assignee: Dave Beech
> Priority: Critical
>
> I tried to write test using mrunit 1.0.0 ( i need to init cache, which is unavaliable in already relased versions)
> when I run my code I get:
> java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
> at org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon(AbstractMockContextWrapper.java:53)
> I read other issues like https://issues.apache.org/jira/browse/MRUNIT-56 and decided to recompile mrunit.
> I've downloaded source, and then:
> mvn install -DskipTests
> Build succes
> mvn install -DskipTests -Dhadoop.version=2
> Build success
> tried both output version with my code:
> mrunit-1.0.0-SNAPSHOT-hadoop1.jar
> mrunit-1.0.0-SNAPSHOT-hadoop2.jar
> but was still getting exception
> I've changed pom.xml specificHadoopVersion to:
> <specificHadoopVersion>0.23.3</specificHadoopVersion>
> and then:
> mvn clean
> mvn install -DskipTests -Dhadoop.version=2
> and there are compilation errors:
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR] /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[464,24] error: cannot find symbol
> [ERROR] class DistributedCache
> /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[478,24] error: cannot find symbol
> [INFO] 2 errors
> that's because initDistributedCache method uses DistributedCache.addLocalFiles(), which is unavaliable in Hadoop 0.23.3
> I tried to fix it, but I'm new to hadoop ant havent succeed yet, any help, please?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-159) MRUnit not compiling with hadoop
0.23.3
Posted by "Jakub (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494726#comment-13494726 ]
Jakub commented on MRUNIT-159:
------------------------------
I've got workaround:
I completely removed content of TestDriver.initDistributedCache(),
and
Instead of using:
URI[] localURIs = context.getCacheFiles();
I do:
URI[] localURIs = StringUtils.stringToURI(context.getConfiguration().getStrings(MRJobConfig.CACHE_FILES));
, and
when I initalize my cache witch :
mrd.withCacheFile(new URI("file:///path/to/dict.txt"));
I tried adding in org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon something like:
try{
when(context.getCacheFiles()).thenAnswer(new Answer<URI[]>() {
@Override
public URI[] answer(final InvocationOnMock invocation) throws IOException {
return DistributedCache.getCacheFiles(contextDriver.getConfiguration());
}
});
} catch (IOException e) {
throw new RuntimeException(e);
}
but it doesn't help
> MRUnit not compiling with hadoop 0.23.3
> ---------------------------------------
>
> Key: MRUNIT-159
> URL: https://issues.apache.org/jira/browse/MRUNIT-159
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jakub
> Priority: Critical
>
> I tried to write test using mrunit 1.0.0 ( i need to init cache, which is unavaliable in already relased versions)
> when I run my code I get:
> java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
> at org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon(AbstractMockContextWrapper.java:53)
> I read other issues like https://issues.apache.org/jira/browse/MRUNIT-56 and decided to recompile mrunit.
> I've downloaded source, and then:
> mvn install -DskipTests
> Build succes
> mvn install -DskipTests -Dhadoop.version=2
> Build success
> tried both output version with my code:
> mrunit-1.0.0-SNAPSHOT-hadoop1.jar
> mrunit-1.0.0-SNAPSHOT-hadoop2.jar
> but was still getting exception
> I've changed pom.xml specificHadoopVersion to:
> <specificHadoopVersion>0.23.3</specificHadoopVersion>
> and then:
> mvn clean
> mvn install -DskipTests -Dhadoop.version=2
> and there are compilation errors:
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR] /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[464,24] error: cannot find symbol
> [ERROR] class DistributedCache
> /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[478,24] error: cannot find symbol
> [INFO] 2 errors
> that's because initDistributedCache method uses DistributedCache.addLocalFiles(), which is unavaliable in Hadoop 0.23.3
> I tried to fix it, but I'm new to hadoop ant havent succeed yet, any help, please?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MRUNIT-159) MRUnit not compiling with hadoop
0.23.3
Posted by "Dave Beech (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dave Beech reassigned MRUNIT-159:
---------------------------------
Assignee: Dave Beech
> MRUnit not compiling with hadoop 0.23.3
> ---------------------------------------
>
> Key: MRUNIT-159
> URL: https://issues.apache.org/jira/browse/MRUNIT-159
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jakub
> Assignee: Dave Beech
> Priority: Critical
>
> I tried to write test using mrunit 1.0.0 ( i need to init cache, which is unavaliable in already relased versions)
> when I run my code I get:
> java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
> at org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon(AbstractMockContextWrapper.java:53)
> I read other issues like https://issues.apache.org/jira/browse/MRUNIT-56 and decided to recompile mrunit.
> I've downloaded source, and then:
> mvn install -DskipTests
> Build succes
> mvn install -DskipTests -Dhadoop.version=2
> Build success
> tried both output version with my code:
> mrunit-1.0.0-SNAPSHOT-hadoop1.jar
> mrunit-1.0.0-SNAPSHOT-hadoop2.jar
> but was still getting exception
> I've changed pom.xml specificHadoopVersion to:
> <specificHadoopVersion>0.23.3</specificHadoopVersion>
> and then:
> mvn clean
> mvn install -DskipTests -Dhadoop.version=2
> and there are compilation errors:
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR] /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[464,24] error: cannot find symbol
> [ERROR] class DistributedCache
> /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[478,24] error: cannot find symbol
> [INFO] 2 errors
> that's because initDistributedCache method uses DistributedCache.addLocalFiles(), which is unavaliable in Hadoop 0.23.3
> I tried to fix it, but I'm new to hadoop ant havent succeed yet, any help, please?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-159) MRUnit not compiling with hadoop
0.23.3
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494869#comment-13494869 ]
Hudson commented on MRUNIT-159:
-------------------------------
Integrated in mrunit-trunk #432 (See [https://builds.apache.org/job/mrunit-trunk/432/])
MRUNIT-159: MRUnit not compiling with hadoop 0.23.3 (Revision 27b8505da44394456f4ae3d91220683e2e5977b5)
Result = SUCCESS
dbeech : https://git-wip-us.apache.org/repos/asf?p=mrunit.git&a=commit&h=27b8505da44394456f4ae3d91220683e2e5977b5
Files :
* src/main/java/org/apache/hadoop/mrunit/internal/util/DistCacheUtils.java
* src/main/java/org/apache/hadoop/mrunit/TestDriver.java
> MRUnit not compiling with hadoop 0.23.3
> ---------------------------------------
>
> Key: MRUNIT-159
> URL: https://issues.apache.org/jira/browse/MRUNIT-159
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jakub
> Assignee: Dave Beech
> Priority: Critical
>
> I tried to write test using mrunit 1.0.0 ( i need to init cache, which is unavaliable in already relased versions)
> when I run my code I get:
> java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
> at org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon(AbstractMockContextWrapper.java:53)
> I read other issues like https://issues.apache.org/jira/browse/MRUNIT-56 and decided to recompile mrunit.
> I've downloaded source, and then:
> mvn install -DskipTests
> Build succes
> mvn install -DskipTests -Dhadoop.version=2
> Build success
> tried both output version with my code:
> mrunit-1.0.0-SNAPSHOT-hadoop1.jar
> mrunit-1.0.0-SNAPSHOT-hadoop2.jar
> but was still getting exception
> I've changed pom.xml specificHadoopVersion to:
> <specificHadoopVersion>0.23.3</specificHadoopVersion>
> and then:
> mvn clean
> mvn install -DskipTests -Dhadoop.version=2
> and there are compilation errors:
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR] /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[464,24] error: cannot find symbol
> [ERROR] class DistributedCache
> /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[478,24] error: cannot find symbol
> [INFO] 2 errors
> that's because initDistributedCache method uses DistributedCache.addLocalFiles(), which is unavaliable in Hadoop 0.23.3
> I tried to fix it, but I'm new to hadoop ant havent succeed yet, any help, please?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MRUNIT-159) MRUnit not compiling with hadoop
0.23.3
Posted by "Jakub (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakub resolved MRUNIT-159.
--------------------------
Resolution: Fixed
Fix Version/s: 1.0.0
> MRUnit not compiling with hadoop 0.23.3
> ---------------------------------------
>
> Key: MRUNIT-159
> URL: https://issues.apache.org/jira/browse/MRUNIT-159
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jakub
> Assignee: Dave Beech
> Priority: Critical
> Fix For: 1.0.0
>
>
> I tried to write test using mrunit 1.0.0 ( i need to init cache, which is unavaliable in already relased versions)
> when I run my code I get:
> java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
> at org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon(AbstractMockContextWrapper.java:53)
> I read other issues like https://issues.apache.org/jira/browse/MRUNIT-56 and decided to recompile mrunit.
> I've downloaded source, and then:
> mvn install -DskipTests
> Build succes
> mvn install -DskipTests -Dhadoop.version=2
> Build success
> tried both output version with my code:
> mrunit-1.0.0-SNAPSHOT-hadoop1.jar
> mrunit-1.0.0-SNAPSHOT-hadoop2.jar
> but was still getting exception
> I've changed pom.xml specificHadoopVersion to:
> <specificHadoopVersion>0.23.3</specificHadoopVersion>
> and then:
> mvn clean
> mvn install -DskipTests -Dhadoop.version=2
> and there are compilation errors:
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR] /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[464,24] error: cannot find symbol
> [ERROR] class DistributedCache
> /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[478,24] error: cannot find symbol
> [INFO] 2 errors
> that's because initDistributedCache method uses DistributedCache.addLocalFiles(), which is unavaliable in Hadoop 0.23.3
> I tried to fix it, but I'm new to hadoop ant havent succeed yet, any help, please?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-159) MRUnit not compiling with hadoop
0.23.3
Posted by "Jakub (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495402#comment-13495402 ]
Jakub commented on MRUNIT-159:
------------------------------
Yes, it solves my issues, thanks.
> MRUnit not compiling with hadoop 0.23.3
> ---------------------------------------
>
> Key: MRUNIT-159
> URL: https://issues.apache.org/jira/browse/MRUNIT-159
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jakub
> Assignee: Dave Beech
> Priority: Critical
> Fix For: 1.0.0
>
>
> I tried to write test using mrunit 1.0.0 ( i need to init cache, which is unavaliable in already relased versions)
> when I run my code I get:
> java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
> at org.apache.hadoop.mrunit.internal.mapreduce.AbstractMockContextWrapper.createCommon(AbstractMockContextWrapper.java:53)
> I read other issues like https://issues.apache.org/jira/browse/MRUNIT-56 and decided to recompile mrunit.
> I've downloaded source, and then:
> mvn install -DskipTests
> Build succes
> mvn install -DskipTests -Dhadoop.version=2
> Build success
> tried both output version with my code:
> mrunit-1.0.0-SNAPSHOT-hadoop1.jar
> mrunit-1.0.0-SNAPSHOT-hadoop2.jar
> but was still getting exception
> I've changed pom.xml specificHadoopVersion to:
> <specificHadoopVersion>0.23.3</specificHadoopVersion>
> and then:
> mvn clean
> mvn install -DskipTests -Dhadoop.version=2
> and there are compilation errors:
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR] /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[464,24] error: cannot find symbol
> [ERROR] class DistributedCache
> /home/kuba/studia/magisterka/mrunit/src/main/java/org/apache/hadoop/mrunit/TestDriver.java:[478,24] error: cannot find symbol
> [INFO] 2 errors
> that's because initDistributedCache method uses DistributedCache.addLocalFiles(), which is unavaliable in Hadoop 0.23.3
> I tried to fix it, but I'm new to hadoop ant havent succeed yet, any help, please?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira