You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mrunit.apache.org by "Jim Donofrio (JIRA)" <ji...@apache.org> on 2012/04/23 04:25:33 UTC
[jira] [Updated] (MRUNIT-105) java serialization tests fail because
they do not set a group comparator
[ https://issues.apache.org/jira/browse/MRUNIT-105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Donofrio updated MRUNIT-105:
--------------------------------
Description:
>From http://mail-archives.apache.org/mod_mbox/incubator-mrunit-dev/201204.mbox/%3C4F927519.2040207%40gmx.net%3E
java.lang.ClassCastException: java.lang.Integer cannot be cast to
org.apache.hadoop.io.WritableComparable
at
org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:119)
at java.util.TreeMap.compare(TreeMap.java:1188)
at java.util.TreeMap.put(TreeMap.java:531)
at
org.apache.hadoop.mrunit.MapReduceDriverBase.shuffle(MapReduceDriverBase.java:172)
at org.apache.hadoop.mrunit.MapReduceDriver.run(MapReduceDriver.java:328)
at
org.apache.hadoop.mrunit.MapReduceDriverBase.runTest(MapReduceDriverBase.java:137)
at org.apache.hadoop.mrunit.TestDriver.runTest(TestDriver.java:158)
at
org.apache.hadoop.mrunit.TestMapReduceDriver.testJavaSerialization(TestMapReduceDriver.java:409)
public int compare(Object a, Object b) {
return compare((WritableComparable)a, (WritableComparable)b);
}
Java 7:
Entry<K,V> t = root;
if (t == null) {
compare(key, key); // type (and possibly null) check
root = new Entry<>(key, value, null);
size = 1;
modCount++;
return null;
}
Java 6:
Entry<K,V> t = root;
if (t == null) {
// TBD:
// 5045147: (coll) Adding null to an empty TreeSet should
// throw NullPointerException
//
// compare(key, key); // type check
root = new Entry<K,V>(key, value, null);
size = 1;
modCount++;
return null;
}
Oops this is a bug in test case not the actual code, good catch. This
problem is not unique to Java 7. The problem only appears in Java 7 because
the java serialization tests only use 1 input value. Java 6's put method in
TreeMap does not call compare when adding to an empty map while Java 7's put
method does call compare when adding to an empty map.
was:
>From http://mail-archives.apache.org/mod_mbox/incubator-mrunit-dev/201204.mbox/%3C4F927519.2040207%40gmx.net%3E
java.lang.ClassCastException: java.lang.Integer cannot be cast to
org.apache.hadoop.io.WritableComparable
at
org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:119)
at java.util.TreeMap.compare(TreeMap.java:1188)
at java.util.TreeMap.put(TreeMap.java:531)
at
org.apache.hadoop.mrunit.MapReduceDriverBase.shuffle(MapReduceDriverBase.java:172)
at org.apache.hadoop.mrunit.MapReduceDriver.run(MapReduceDriver.java:328)
at
org.apache.hadoop.mrunit.MapReduceDriverBase.runTest(MapReduceDriverBase.java:137)
at org.apache.hadoop.mrunit.TestDriver.runTest(TestDriver.java:158)
at
org.apache.hadoop.mrunit.TestMapReduceDriver.testJavaSerialization(TestMapReduceDriver.java:409)
public int compare(Object a, Object b) {
return compare((WritableComparable)a, (WritableComparable)b);
}
Java 7:
Entry<K,V> t = root;
if (t == null) {
compare(key, key); // type (and possibly null) check
root = new Entry<>(key, value, null);
size = 1;
modCount++;
return null;
}
Java 6:
Entry<K,V> t = root;
if (t == null) {
// TBD:
// 5045147: (coll) Adding null to an empty TreeSet should
// throw NullPointerException
//
// compare(key, key); // type check
root = new Entry<K,V>(key, value, null);
size = 1;
modCount++;
return null;
}
Oops this is a bug in test case not the actual code, good catch. This
problem is not unique to Java 7. The problem only appears in Java 7 because
my java serialization tests only use 1 input value. Java 6's put method in
TreeMap does not call compare when adding to an empty map while Java 7's put
method does call compare when adding to an empty map.
This can be fixed in the test case by setting a
OutputValueGroupingComparator in the conf for Integer. Users that use java
or other non Writable serializations would know they have to set the
relevant comparators.
Summary: java serialization tests fail because they do not set a group comparator (was: java serialization tests fail to set group comparator)
> java serialization tests fail because they do not set a group comparator
> ------------------------------------------------------------------------
>
> Key: MRUNIT-105
> URL: https://issues.apache.org/jira/browse/MRUNIT-105
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Jim Donofrio
> Assignee: Jim Donofrio
> Fix For: 1.0.0
>
>
> From http://mail-archives.apache.org/mod_mbox/incubator-mrunit-dev/201204.mbox/%3C4F927519.2040207%40gmx.net%3E
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> org.apache.hadoop.io.WritableComparable
> at
> org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:119)
> at java.util.TreeMap.compare(TreeMap.java:1188)
> at java.util.TreeMap.put(TreeMap.java:531)
> at
> org.apache.hadoop.mrunit.MapReduceDriverBase.shuffle(MapReduceDriverBase.java:172)
> at org.apache.hadoop.mrunit.MapReduceDriver.run(MapReduceDriver.java:328)
> at
> org.apache.hadoop.mrunit.MapReduceDriverBase.runTest(MapReduceDriverBase.java:137)
> at org.apache.hadoop.mrunit.TestDriver.runTest(TestDriver.java:158)
> at
> org.apache.hadoop.mrunit.TestMapReduceDriver.testJavaSerialization(TestMapReduceDriver.java:409)
> public int compare(Object a, Object b) {
> return compare((WritableComparable)a, (WritableComparable)b);
> }
> Java 7:
> Entry<K,V> t = root;
> if (t == null) {
> compare(key, key); // type (and possibly null) check
> root = new Entry<>(key, value, null);
> size = 1;
> modCount++;
> return null;
> }
> Java 6:
> Entry<K,V> t = root;
> if (t == null) {
> // TBD:
> // 5045147: (coll) Adding null to an empty TreeSet should
> // throw NullPointerException
> //
> // compare(key, key); // type check
> root = new Entry<K,V>(key, value, null);
> size = 1;
> modCount++;
> return null;
> }
> Oops this is a bug in test case not the actual code, good catch. This
> problem is not unique to Java 7. The problem only appears in Java 7 because
> the java serialization tests only use 1 input value. Java 6's put method in
> TreeMap does not call compare when adding to an empty map while Java 7's put
> method does call compare when adding to an empty map.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira