You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2014/10/15 02:55:30 UTC
Review Request 26722: hive command line fails to start after setting
hive.execution.engine to tez in HDP 2.2.1
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26722/
-----------------------------------------------------------
Review request for Ambari, Sumit Mohanty and Sid Wagle.
Bugs: AMBARI-7739
https://issues.apache.org/jira/browse/AMBARI-7739
Repository: ambari
Description
-------
Patch on top of change already committed.
Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, Hive, Zookeeper.
After switching the hive.execution.engine from "mr" to "tez", attempting to run the hive shell generates an error.
Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
... 8 more
In the original patch, https://reviews.apache.org/r/26591/ , the hadoop-env is stored as xml and needed to escpae && as && but it ends up appearing in that same way in the hadoop-env.sh file, which prevents it from working.
So intead of using &&, use nested ifs.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 5d70420
Diff: https://reviews.apache.org/r/26722/diff/
Testing
-------
Deployed a new cluster with the changes to hadoop-env.xml and verified that hive on tez was able to start.
I did need to run the following first,
hadoop fs -mkdir /apps/tez
hadoop fs -copyFromLocal /usr/hdp/current/tez-client/lib/tez-0.6.0.2.2.0.0-913.tar.gz /apps/tez/tez.tar.gz
hadoop fs -ls /apps/tez/
Thanks,
Alejandro Fernandez
Re: Review Request 26722: hive command line fails to start after
setting hive.execution.engine to tez in HDP 2.2.1
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26722/#review56629
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml
<https://reviews.apache.org/r/26722/#comment97015>
Clarify if the class path should not need path to "/usr/hdp/current/tez-client" after the latest version of tez.tar.gz is uploaded.
- Sumit Mohanty
On Oct. 15, 2014, 12:59 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26722/
> -----------------------------------------------------------
>
> (Updated Oct. 15, 2014, 12:59 a.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Sid Wagle.
>
>
> Bugs: AMBARI-7739
> https://issues.apache.org/jira/browse/AMBARI-7739
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Patch on top of change already committed.
>
> Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, Hive, Zookeeper.
> After switching the hive.execution.engine from "mr" to "tez", attempting to run the hive shell generates an error.
> Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
> Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598)
> at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206)
> at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
> ... 8 more
>
>
> In the original patch, https://reviews.apache.org/r/26591/ , the hadoop-env is stored as xml and needed to escpae && as && but it ends up appearing in that same way in the hadoop-env.sh file, which prevents it from working.
> So intead of using &&, use nested ifs.
>
> **This needs two +1s in order to be committed into branch-1.7.0**
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 5d70420
>
> Diff: https://reviews.apache.org/r/26722/diff/
>
>
> Testing
> -------
>
> Deployed a new cluster with the changes to hadoop-env.xml and verified that hive on tez was able to start.
> I did need to run the following first,
>
> hadoop fs -mkdir /apps/tez
> hadoop fs -copyFromLocal /usr/hdp/current/tez-client/lib/tez-0.6.0.2.2.0.0-913.tar.gz /apps/tez/tez.tar.gz
> hadoop fs -ls /apps/tez/
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 26722: hive command line fails to start after
setting hive.execution.engine to tez in HDP 2.2.1
Posted by Jaimin Jetly <ja...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26722/#review56635
-----------------------------------------------------------
Ship it!
Ship It!
- Jaimin Jetly
On Oct. 15, 2014, 1:09 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26722/
> -----------------------------------------------------------
>
> (Updated Oct. 15, 2014, 1:09 a.m.)
>
>
> Review request for Ambari, Hitesh Shah, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-7739
> https://issues.apache.org/jira/browse/AMBARI-7739
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Patch on top of change already committed.
>
> Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, Hive, Zookeeper.
> After switching the hive.execution.engine from "mr" to "tez", attempting to run the hive shell generates an error.
> Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
> Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598)
> at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206)
> at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
> ... 8 more
>
>
> In the original patch, https://reviews.apache.org/r/26591/ , the hadoop-env is stored as xml and needed to escpae && as && but it ends up appearing in that same way in the hadoop-env.sh file, which prevents it from working.
> So intead of using &&, use nested ifs.
>
> **This needs two +1s in order to be committed into branch-1.7.0**
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 5d70420
>
> Diff: https://reviews.apache.org/r/26722/diff/
>
>
> Testing
> -------
>
> Deployed a new cluster with the changes to hadoop-env.xml and verified that hive on tez was able to start.
> I did need to run the following first,
>
> hadoop fs -mkdir /apps/tez
> hadoop fs -copyFromLocal /usr/hdp/current/tez-client/lib/tez-0.6.0.2.2.0.0-913.tar.gz /apps/tez/tez.tar.gz
> hadoop fs -ls /apps/tez/
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 26722: hive command line fails to start after
setting hive.execution.engine to tez in HDP 2.2.1
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26722/
-----------------------------------------------------------
(Updated Oct. 15, 2014, 1:09 a.m.)
Review request for Ambari, Hitesh Shah, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-7739
https://issues.apache.org/jira/browse/AMBARI-7739
Repository: ambari
Description
-------
Patch on top of change already committed.
Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, Hive, Zookeeper.
After switching the hive.execution.engine from "mr" to "tez", attempting to run the hive shell generates an error.
Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
... 8 more
In the original patch, https://reviews.apache.org/r/26591/ , the hadoop-env is stored as xml and needed to escpae && as && but it ends up appearing in that same way in the hadoop-env.sh file, which prevents it from working.
So intead of using &&, use nested ifs.
**This needs two +1s in order to be committed into branch-1.7.0**
Diffs (updated)
-----
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 5d70420
Diff: https://reviews.apache.org/r/26722/diff/
Testing
-------
Deployed a new cluster with the changes to hadoop-env.xml and verified that hive on tez was able to start.
I did need to run the following first,
hadoop fs -mkdir /apps/tez
hadoop fs -copyFromLocal /usr/hdp/current/tez-client/lib/tez-0.6.0.2.2.0.0-913.tar.gz /apps/tez/tez.tar.gz
hadoop fs -ls /apps/tez/
Thanks,
Alejandro Fernandez
Re: Review Request 26722: hive command line fails to start after
setting hive.execution.engine to tez in HDP 2.2.1
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26722/
-----------------------------------------------------------
(Updated Oct. 15, 2014, 1:09 a.m.)
Review request for Ambari, Hitesh Shah, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-7739
https://issues.apache.org/jira/browse/AMBARI-7739
Repository: ambari
Description
-------
Patch on top of change already committed.
Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, Hive, Zookeeper.
After switching the hive.execution.engine from "mr" to "tez", attempting to run the hive shell generates an error.
Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
... 8 more
In the original patch, https://reviews.apache.org/r/26591/ , the hadoop-env is stored as xml and needed to escpae && as && but it ends up appearing in that same way in the hadoop-env.sh file, which prevents it from working.
So intead of using &&, use nested ifs.
**This needs two +1s in order to be committed into branch-1.7.0**
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 5d70420
Diff: https://reviews.apache.org/r/26722/diff/
Testing
-------
Deployed a new cluster with the changes to hadoop-env.xml and verified that hive on tez was able to start.
I did need to run the following first,
hadoop fs -mkdir /apps/tez
hadoop fs -copyFromLocal /usr/hdp/current/tez-client/lib/tez-0.6.0.2.2.0.0-913.tar.gz /apps/tez/tez.tar.gz
hadoop fs -ls /apps/tez/
Thanks,
Alejandro Fernandez
Re: Review Request 26722: hive command line fails to start after
setting hive.execution.engine to tez in HDP 2.2.1
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26722/#review56630
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On Oct. 15, 2014, 12:59 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26722/
> -----------------------------------------------------------
>
> (Updated Oct. 15, 2014, 12:59 a.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Sid Wagle.
>
>
> Bugs: AMBARI-7739
> https://issues.apache.org/jira/browse/AMBARI-7739
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Patch on top of change already committed.
>
> Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, Hive, Zookeeper.
> After switching the hive.execution.engine from "mr" to "tez", attempting to run the hive shell generates an error.
> Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
> Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598)
> at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206)
> at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
> ... 8 more
>
>
> In the original patch, https://reviews.apache.org/r/26591/ , the hadoop-env is stored as xml and needed to escpae && as && but it ends up appearing in that same way in the hadoop-env.sh file, which prevents it from working.
> So intead of using &&, use nested ifs.
>
> **This needs two +1s in order to be committed into branch-1.7.0**
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 5d70420
>
> Diff: https://reviews.apache.org/r/26722/diff/
>
>
> Testing
> -------
>
> Deployed a new cluster with the changes to hadoop-env.xml and verified that hive on tez was able to start.
> I did need to run the following first,
>
> hadoop fs -mkdir /apps/tez
> hadoop fs -copyFromLocal /usr/hdp/current/tez-client/lib/tez-0.6.0.2.2.0.0-913.tar.gz /apps/tez/tez.tar.gz
> hadoop fs -ls /apps/tez/
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 26722: hive command line fails to start after
setting hive.execution.engine to tez in HDP 2.2.1
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26722/
-----------------------------------------------------------
(Updated Oct. 15, 2014, 12:59 a.m.)
Review request for Ambari, Sumit Mohanty and Sid Wagle.
Bugs: AMBARI-7739
https://issues.apache.org/jira/browse/AMBARI-7739
Repository: ambari
Description (updated)
-------
Patch on top of change already committed.
Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, Hive, Zookeeper.
After switching the hive.execution.engine from "mr" to "tez", attempting to run the hive shell generates an error.
Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
... 8 more
In the original patch, https://reviews.apache.org/r/26591/ , the hadoop-env is stored as xml and needed to escpae && as && but it ends up appearing in that same way in the hadoop-env.sh file, which prevents it from working.
So intead of using &&, use nested ifs.
**This needs two +1s in order to be committed into branch-1.7.0**
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 5d70420
Diff: https://reviews.apache.org/r/26722/diff/
Testing
-------
Deployed a new cluster with the changes to hadoop-env.xml and verified that hive on tez was able to start.
I did need to run the following first,
hadoop fs -mkdir /apps/tez
hadoop fs -copyFromLocal /usr/hdp/current/tez-client/lib/tez-0.6.0.2.2.0.0-913.tar.gz /apps/tez/tez.tar.gz
hadoop fs -ls /apps/tez/
Thanks,
Alejandro Fernandez