You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Devaraj Das (Created) (JIRA)" <ji...@apache.org> on 2012/03/20 18:25:38 UTC
[jira] [Created] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Fix unit test failures with IBM's JDK
-------------------------------------
Key: HADOOP-8192
URL: https://issues.apache.org/jira/browse/HADOOP-8192
Project: Hadoop Common
Issue Type: Bug
Environment: java version "1.6.0"
Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
J9VM - 20111207_096808
JIT - r9_20111107_21307ifx1
GC - 20110519_AA)
JCL - 20111104_02
Reporter: Devaraj Das
Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Luke Lu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240503#comment-13240503 ]
Luke Lu commented on HADOOP-8192:
---------------------------------
@Kumar, Sorry, faulty memory :) I really meant TestStreamingBadRecords only fails on plinux after the compliance bug fix.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Kumar Ravi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236990#comment-13236990 ]
Kumar Ravi commented on HADOOP-8192:
------------------------------------
While debugging this issue it was observed that the order by which the racksToBlocks HashMap gets populated seems to matter. As per Robert Evans and Devaraj Das, it appears by design that the order should not matter.
The reason order happens to play a role here is that as soon as all the blocks are accounted for, getMoreSplits() stops iterating through the racks, and depending upon which rack(s) each block is replicated on, and depending upon when each rack is processed in the loop within getMoreSplits(), one can end up with different split counts, and as a result fail the testcase in some situations.
Specifically for this testcase, there are 3 racks that are simulated where each of these 3 racks have a datanode each. Datanode 1 has replicas of all the blocks of all the 3 files (file1, file2, and file3) while Datanode 2 has all the blocks of files file2 and file 3 and Datanode 3 has all the blocks of only file3. As soon as Rack 1 is processed, getMoreSplits() exits with a split count of the number of times it stays in this loop. So in this scenario, if Rack1 gets processed last, one will end up with a split count of 3. If Rack1 gets processed in the beginning, split count will be 1. The testcase is expecting a return value of 3 which is the value returned if running on Sun JVM, but a value of 1 or 2 may be returned depending on when rack1 gets processed.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Kumar Ravi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239590#comment-13239590 ]
Kumar Ravi commented on HADOOP-8192:
------------------------------------
Luke - I need to correct a statement you made. These tests fail on IBM Java 6 on x86_64 as well, and the reason for the failure is as described in my earlier comment. So this is an IBM Java vs Sun JVM behavior rather than a plinux one.
I would still like to get a better understanding than just code comments - If getSplits() method in CombineFileInputFormat is working as designed. Can you suggest someone who has expertise on this?
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Devaraj Das (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237262#comment-13237262 ]
Devaraj Das commented on HADOOP-8192:
-------------------------------------
Basically somehow make the order of inserts/gets consistent across JVMs by using appropriate datastructures...
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Kumar Ravi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240507#comment-13240507 ]
Kumar Ravi commented on HADOOP-8192:
------------------------------------
@Luke, Thx. -- I am not sure what you mean by the compliance bug fix.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Amir Sanjar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238659#comment-13238659 ]
Amir Sanjar commented on HADOOP-8192:
-------------------------------------
Luke, please explain what you mean by "I suggest that we split the JIRA into 3 separate ones, as they are technically unrelated"..
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Kumar Ravi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238473#comment-13238473 ]
Kumar Ravi commented on HADOOP-8192:
------------------------------------
I think it would be extremely useful to understand if CombineFileInputFormat's getSplits() is working as designed -- Should the method exit out of the loop as soon as all blocks are accounted for. If so, the next question is - Is this is a testcase scenario issue? -- In a production environment are there scenarios where a Datanode will contain all the blocks in a cluster?
We would like to make sure that we are not fixing a non-existent problem to pass a test case, and inadvertently break the split functionality when it is working without issues. If this just happens to be a behavior issue between two JVMs for a given scenario then it might be more prudent to create separate test paths for each JVM.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Luke Lu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239317#comment-13239317 ]
Luke Lu commented on HADOOP-8192:
---------------------------------
@Amir, 3 test failures have different root causes and require different patch review cycles. E.g., TestCombineFileInputFormat failure is a result of relying on undefined HashMap iteration order in the implementation of CombineFileInputFormat; TestStreamingBadRecords, AFAICT, only fails on plinux (you guys need to figure out the root cause for this); TestCapacityScheduler failures is a bit more serious, as it expose a JLS compliance bug (that can crash JobTracker) in the current code base as well as the HashMap order issue. Combining all these together is confusing for the patch review process.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Luke Lu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238585#comment-13238585 ]
Luke Lu commented on HADOOP-8192:
---------------------------------
I suggest that we split the JIRA into 3 separate ones, as they are technically unrelated, each requiring patches for trunk and branch-1. I might have some cycles to work on this in the coming week.
For CombineFileInputFormat, I'm -1 on creating separately test paths for each JVM, as it's a maintenance nightmare.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Devaraj Das (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237260#comment-13237260 ]
Devaraj Das commented on HADOOP-8192:
-------------------------------------
One option is to make the relevant hashmaps sorted in CombineFileInputFormat, and then fix the testcase to check the correct (and consistent) values in the asserts. Would that fix the testcase problem? The other option is to make the testcase more robust so that it can tolerate the fact that ordering could be different on different JVMs.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8192) Fix unit test failures with IBM's
JDK
Posted by "Luke Lu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240520#comment-13240520 ]
Luke Lu commented on HADOOP-8192:
---------------------------------
I meant JLS (Java Language Specification) compliance, which mandates certain properties for equality and comparison.
> Fix unit test failures with IBM's JDK
> -------------------------------------
>
> Key: HADOOP-8192
> URL: https://issues.apache.org/jira/browse/HADOOP-8192
> Project: Hadoop Common
> Issue Type: Bug
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT - r9_20111107_21307ifx1
> GC - 20110519_AA)
> JCL - 20111104_02
> Reporter: Devaraj Das
>
> Some tests fail with IBM's JDK. They are org.apache.hadoop.mapred.lib.TestCombineFileInputFormat, org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat, org.apache.hadoop.streaming.TestStreamingBadRecords, org.apache.hadoop.mapred.TestCapacityScheduler. This jira is to track fixing these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira