You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2015/02/23 18:00:18 UTC
[jira] [Commented] (HADOOP-11617) Several tests are not stable (on
OpenJDK / x86_64 / RHEL 7.1)
[ https://issues.apache.org/jira/browse/HADOOP-11617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14333504#comment-14333504 ]
Allen Wittenauer commented on HADOOP-11617:
-------------------------------------------
First, some JIRA etiquette:
Please cut back on how much stuff you have listed in the description. Every comment to this JIRA is going to send the entirety of the description to everyone. Either put it in a comment or as an attachement.
Second, I don't think anyone has spent any time on actually make sure OpenJDK works on modern releases, so it isn't too surprising that some tests fail. Although http://wiki.apache.org/hadoop/HadoopJavaVersions claims Hortonworks has done such tests, it appears to be with both a very old version of OpenJDK and a very old version of Hadoop. I'm not sure I'd put any faith in any of those claims anymore. Probably time to update that page.
> Several tests are not stable (on OpenJDK / x86_64 / RHEL 7.1)
> -------------------------------------------------------------
>
> Key: HADOOP-11617
> URL: https://issues.apache.org/jira/browse/HADOOP-11617
> Project: Hadoop Common
> Issue Type: Bug
> Environment: RHEL 7.1 / x86_64
> Reporter: Tony Reix
>
> I'm now porting Hadoop 2.4.1 on PPC64 HW.
> In order to have an idea about the stability of the tests, I've run several times all Hadoop tests in a RHEL 7.1 / x86_64 / OpenJDK environment.
> I've run it 12 times, for now, on the EXACT same environment.
> Using some tool, I have extracted the list of tests that do not always produce the same results, listed below, showing some test unstability.
> I 'm now doing the same for version 2.6.0 on OpenJDK/Ubuntu/x86_64.
> Report gives:
> test-name N: T F E S |
> where:
> - N is the number of the test log file
> - and:
> T: Tests run
> F: Failure
> E: Error
> S: Skipped
> As an example (see below for full data):
> hdfs.server.datanode.TestDeleteBlockPool 0: 2 0 0 0 |
> ........
> hdfs.server.datanode.TestDeleteBlockPool 4: 2 0 1 0 |
> ........
> hdfs.server.datanode.TestDeleteBlockPool 11: 2 0 0 0 |
> means that this test, out of 12 runs, had an error once, with 0 Failure and 0 Skipped test.
> # ./BuildUnstable.sh -A mvn.Test.OpenJDK.res00 mvn.Test.OpenJDK.res00a mvn.Test.OpenJDK.res01a mvn.Test.OpenJDK.res01 mvn.Test.OpenJDK.res02 mvn.Test.OpenJDK.res03 mvn.Test.OpenJDK.res04 mvn.Test.OpenJDK.res05 mvn.Test.OpenJDK.res06 mvn.Test.OpenJDK.res07 mvn.Test.OpenJDK.res08 mvn.Test.OpenJDK.res09
> Files:
> 0: mvn.Test.OpenJDK.res00
> 1: mvn.Test.OpenJDK.res00a
> 2: mvn.Test.OpenJDK.res01a
> 3: mvn.Test.OpenJDK.res01
> 4: mvn.Test.OpenJDK.res02
> 5: mvn.Test.OpenJDK.res03
> 6: mvn.Test.OpenJDK.res04
> 7: mvn.Test.OpenJDK.res05
> 8: mvn.Test.OpenJDK.res06
> 9: mvn.Test.OpenJDK.res07
> 10: mvn.Test.OpenJDK.res08
> 11: mvn.Test.OpenJDK.res09
> Unstable Tests:
> ------------------------------------------------------------ T F E S |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 0: 3 0 2 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 1: 3 0 0 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 2: 3 0 1 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 3: 3 0 2 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 4: 3 0 1 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 5: 3 0 2 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 6: 3 0 1 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 7: 3 0 1 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 8: 3 0 0 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 9: 3 0 1 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 10: 3 0 0 0 |
> hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 11: 3 0 1 0 |
> ------------------------------------------------------------ T F E S |
> mapreduce.v2.app.TestRMContainerAllocator 0: |
> mapreduce.v2.app.TestRMContainerAllocator 1: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 2: |
> mapreduce.v2.app.TestRMContainerAllocator 3: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 4: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 5: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 6: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 7: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 8: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 9: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 10: 15 0 0 0 |
> mapreduce.v2.app.TestRMContainerAllocator 11: 15 0 0 0 |
> ------------------------------------------------------------ T F E S |
> hdfs.server.datanode.TestDeleteBlockPool 0: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 1: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 2: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 3: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 4: 2 0 1 0 |
> hdfs.server.datanode.TestDeleteBlockPool 5: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 6: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 7: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 8: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 9: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 10: 2 0 0 0 |
> hdfs.server.datanode.TestDeleteBlockPool 11: 2 0 0 0 |
> ------------------------------------------------------------ T F E S |
> http.TestHttpServerLifecycle 0: 7 0 0 0 |
> http.TestHttpServerLifecycle 1: 7 0 0 0 |
> http.TestHttpServerLifecycle 2: |
> http.TestHttpServerLifecycle 3: |
> http.TestHttpServerLifecycle 4: |
> http.TestHttpServerLifecycle 5: 7 0 0 0 |
> http.TestHttpServerLifecycle 6: |
> http.TestHttpServerLifecycle 7: |
> http.TestHttpServerLifecycle 8: 7 0 0 0 |
> http.TestHttpServerLifecycle 9: 7 0 0 0 |
> http.TestHttpServerLifecycle 10: 7 0 0 0 |
> http.TestHttpServerLifecycle 11: 7 0 0 0 |
> ------------------------------------------------------------ T F E S |
> conf.TestConfServlet 0: 2283 106 68 27 |
> conf.TestConfServlet 1: 2283 106 68 27 |
> conf.TestConfServlet 2: 2276 106 68 27 |
> conf.TestConfServlet 3: 2276 110 68 27 |
> conf.TestConfServlet 4: 2276 106 68 27 |
> conf.TestConfServlet 5: 2283 109 68 27 |
> conf.TestConfServlet 6: 2276 109 68 27 |
> conf.TestConfServlet 7: 2276 106 68 27 |
> conf.TestConfServlet 8: 2283 109 68 27 |
> conf.TestConfServlet 9: 2283 110 68 27 |
> conf.TestConfServlet 10: 2283 106 68 27 |
> conf.TestConfServlet 11: 2283 106 68 27 |
> ------------------------------------------------------------ T F E S |
> mapreduce.v2.app.job.impl.TestTaskAttempt 0: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 1: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 2: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 3: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 4: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 5: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 6: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 7: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 8: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 9: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 10: 13 0 0 0 |
> mapreduce.v2.app.job.impl.TestTaskAttempt 11: 13 0 2 0 |
> ------------------------------------------------------------ T F E S |
> mapred.TestLocalContainerLauncher 0: 251 0 2 0 |
> mapred.TestLocalContainerLauncher 1: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 2: 251 0 2 0 |
> mapred.TestLocalContainerLauncher 3: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 4: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 5: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 6: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 7: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 8: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 9: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 10: 266 0 2 0 |
> mapred.TestLocalContainerLauncher 11: 266 0 4 0 |
> ------------------------------------------------------------ T F E S |
> fs.TestLocalDirAllocator 0: 30 6 0 0 |
> fs.TestLocalDirAllocator 1: 30 6 0 0 |
> fs.TestLocalDirAllocator 2: 30 6 0 0 |
> fs.TestLocalDirAllocator 3: 30 9 0 0 |
> fs.TestLocalDirAllocator 4: 30 6 0 0 |
> fs.TestLocalDirAllocator 5: 30 9 0 0 |
> fs.TestLocalDirAllocator 6: 30 9 0 0 |
> fs.TestLocalDirAllocator 7: 30 6 0 0 |
> fs.TestLocalDirAllocator 8: 30 9 0 0 |
> fs.TestLocalDirAllocator 9: 30 9 0 0 |
> fs.TestLocalDirAllocator 10: 30 6 0 0 |
> fs.TestLocalDirAllocator 11: 30 6 0 0 |
> ------------------------------------------------------------ T F E S |
> metrics2.impl.TestMetricsSystemImpl 0: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 1: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 2: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 3: 6 1 0 0 |
> metrics2.impl.TestMetricsSystemImpl 4: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 5: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 6: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 7: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 8: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 9: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 10: 6 0 0 0 |
> metrics2.impl.TestMetricsSystemImpl 11: 6 0 0 0 |
> ------------------------------------------------------------ T F E S |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 0: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 1: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 2: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 3: 58 1 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 4: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 5: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 6: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 7: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 8: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 9: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 10: 58 0 0 0 |
> yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 11: 58 0 0 0 |
> ------------------------------------------------------------ T F E S |
> yarn.server.resourcemanager.TestClientRMTokens 0: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 1: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 2: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 3: 592 2 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 4: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 5: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 6: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 7: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 8: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 9: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 10: 592 1 6 1 |
> yarn.server.resourcemanager.TestClientRMTokens 11: 592 1 6 1 |
> ------------------------------------------------------------ T F E S |
> ipc.TestRPC 0: 18 1 7 0 |
> ipc.TestRPC 1: 18 1 7 0 |
> ipc.TestRPC 2: 18 1 7 0 |
> ipc.TestRPC 3: 18 1 7 0 |
> ipc.TestRPC 4: 18 1 7 0 |
> ipc.TestRPC 5: 18 1 7 0 |
> ipc.TestRPC 6: 18 1 7 0 |
> ipc.TestRPC 7: 18 1 7 0 |
> ipc.TestRPC 8: 18 1 7 0 |
> ipc.TestRPC 9: 18 2 7 0 |
> ipc.TestRPC 10: 18 1 7 0 |
> ipc.TestRPC 11: 18 1 7 0 |
> ------------------------------------------------------------ T F E S |
> net.TestHdfsNetworkTopologyWithNodeGroup 0: 2654 12 15 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 1: 2654 12 13 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 2: 2654 12 14 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 3: 2654 12 15 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 4: 2654 12 15 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 5: 2654 12 15 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 6: 2654 12 14 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 7: 2654 12 14 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 8: 2654 12 13 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 9: 2654 12 14 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 10: 2654 12 13 15 |
> net.TestHdfsNetworkTopologyWithNodeGroup 11: 2654 12 14 15 |
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)