You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2012/08/14 10:15:38 UTC
[jira] [Created] (HIVE-3383) MinimrCliDriver test hangs when run on
20S (aka 1.0) hadoop profile
Ashutosh Chauhan created HIVE-3383:
--------------------------------------
Summary: MinimrCliDriver test hangs when run on 20S (aka 1.0) hadoop profile
Key: HIVE-3383
URL: https://issues.apache.org/jira/browse/HIVE-3383
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Ashutosh Chauhan
I ran
{ant clean package test -Dtestcase=TestMinimrCliDriver -Dhadoop.mr.rev=20S}
This hangs after printing following on console:
{code}
test:
[junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
[junit] Starting DataNode 0 with dfs.data.dir: build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
[junit] Starting DataNode 1 with dfs.data.dir: build/test/data/dfs/data/data3,build/test/data/dfs/data/data4
[junit] Starting DataNode 2 with dfs.data.dir: build/test/data/dfs/data/data5,build/test/data/dfs/data/data6
[junit] Starting DataNode 3 with dfs.data.dir: build/test/data/dfs/data/data7,build/test/data/dfs/data/data8
[junit] Generating rack names for tasktrackers
[junit] Generating host names for tasktrackers
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3383) MinimrCliDriver test hangs when run
on 20S (aka 1.0) hadoop profile
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433995#comment-13433995 ]
Ashutosh Chauhan commented on HIVE-3383:
----------------------------------------
I collected stack traces multiple times while test was in hang state. I always found main thread in following stack:
{code}
"main" prio=10 tid=0x08800000 nid=0x52cf waiting on condition [0xf72fa000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.mapred.MiniMRCluster.startJobTracker(MiniMRCluster.java:638)
at org.apache.hadoop.mapred.MiniMRCluster.startJobTracker(MiniMRCluster.java:609)
at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:507)
at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:462)
at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:454)
at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:446)
at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:436)
at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:426)
at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:417)
at org.apache.hadoop.hive.ql.QTestUtil.<init>(QTestUtil.java:278)
at org.apache.hadoop.hive.cli.TestMinimrCliDriver.<clinit>(TestMinimrCliDriver.java:51)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:373)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
{code}
Whats happening here is JobTracker is in wait state (forever). Further analysis showed that JT makes an RPC call to NN, NN is trying to load a class which it fails to find and then keep on throwing ClassNotFoundException. Then DFSClient (embedded in JT) keeps on retrying and system remains in hang state. Fix is to download the missing jar file (http://mirrors.ibiblio.org/pub/mirrors/maven2/javax/ws/rs/jsr311-api/1.0/jsr311-api-1.0.jar) and place it in ivy/lib/default dir after compilation but before starting test run. I am not sure whether its a bug in a hive build system wherein it should download this jar file or in hadoop build system, where it should have declared this jar as its dependency in its pom file, so that downstream project will pull in this jar while pulling in hadoop and its dependency.
> MinimrCliDriver test hangs when run on 20S (aka 1.0) hadoop profile
> -------------------------------------------------------------------
>
> Key: HIVE-3383
> URL: https://issues.apache.org/jira/browse/HIVE-3383
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Reporter: Ashutosh Chauhan
>
> I ran
> {code}
> ant clean package test -Dtestcase=TestMinimrCliDriver -Dhadoop.mr.rev=20S
> {code}
> This hangs after printing following on console:
> {code}
> test:
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
> [junit] Starting DataNode 0 with dfs.data.dir: build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
> [junit] Starting DataNode 1 with dfs.data.dir: build/test/data/dfs/data/data3,build/test/data/dfs/data/data4
> [junit] Starting DataNode 2 with dfs.data.dir: build/test/data/dfs/data/data5,build/test/data/dfs/data/data6
> [junit] Starting DataNode 3 with dfs.data.dir: build/test/data/dfs/data/data7,build/test/data/dfs/data/data8
> [junit] Generating rack names for tasktrackers
> [junit] Generating host names for tasktrackers
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3383) MinimrCliDriver test hangs when run on
20S (aka 1.0) hadoop profile
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-3383:
-----------------------------------
Description:
I ran
{code}
ant clean package test -Dtestcase=TestMinimrCliDriver -Dhadoop.mr.rev=20S
{code}
This hangs after printing following on console:
{code}
test:
[junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
[junit] Starting DataNode 0 with dfs.data.dir: build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
[junit] Starting DataNode 1 with dfs.data.dir: build/test/data/dfs/data/data3,build/test/data/dfs/data/data4
[junit] Starting DataNode 2 with dfs.data.dir: build/test/data/dfs/data/data5,build/test/data/dfs/data/data6
[junit] Starting DataNode 3 with dfs.data.dir: build/test/data/dfs/data/data7,build/test/data/dfs/data/data8
[junit] Generating rack names for tasktrackers
[junit] Generating host names for tasktrackers
{code}
was:
I ran
{ant clean package test -Dtestcase=TestMinimrCliDriver -Dhadoop.mr.rev=20S}
This hangs after printing following on console:
{code}
test:
[junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
[junit] Starting DataNode 0 with dfs.data.dir: build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
[junit] Starting DataNode 1 with dfs.data.dir: build/test/data/dfs/data/data3,build/test/data/dfs/data/data4
[junit] Starting DataNode 2 with dfs.data.dir: build/test/data/dfs/data/data5,build/test/data/dfs/data/data6
[junit] Starting DataNode 3 with dfs.data.dir: build/test/data/dfs/data/data7,build/test/data/dfs/data/data8
[junit] Generating rack names for tasktrackers
[junit] Generating host names for tasktrackers
{code}
> MinimrCliDriver test hangs when run on 20S (aka 1.0) hadoop profile
> -------------------------------------------------------------------
>
> Key: HIVE-3383
> URL: https://issues.apache.org/jira/browse/HIVE-3383
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Reporter: Ashutosh Chauhan
>
> I ran
> {code}
> ant clean package test -Dtestcase=TestMinimrCliDriver -Dhadoop.mr.rev=20S
> {code}
> This hangs after printing following on console:
> {code}
> test:
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
> [junit] Starting DataNode 0 with dfs.data.dir: build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
> [junit] Starting DataNode 1 with dfs.data.dir: build/test/data/dfs/data/data3,build/test/data/dfs/data/data4
> [junit] Starting DataNode 2 with dfs.data.dir: build/test/data/dfs/data/data5,build/test/data/dfs/data/data6
> [junit] Starting DataNode 3 with dfs.data.dir: build/test/data/dfs/data/data7,build/test/data/dfs/data/data8
> [junit] Generating rack names for tasktrackers
> [junit] Generating host names for tasktrackers
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira