You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Mich Talebzadeh <mi...@peridale.co.uk> on 2016/01/05 01:00:01 UTC
Hive on TEZ fails starting
Hi,
Trying to run Hive on TEZ for the first time. Getting the error below
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.001 seconds)
0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
No rows affected (0.018 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
ERROR : Failed to execute tez graph.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal
character in scheme name at index 0:
"hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
at org.apache.hadoop.fs.Path.initialize(Path.java:206)
at org.apache.hadoop.fs.Path.<init>(Path.java:172)
at
org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(Tez
ClientUtils.java:964)
at
org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:2
28)
at
org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUti
ls.java:203)
at
org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:883)
at org.apache.tez.client.TezClient.start(TezClient.java:365)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java
:184)
at
org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:239)
at
org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.jav
a:154)
at
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.j
ava:71)
at
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java
:206)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
va:1628)
at
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:2
18)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
45)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
15)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.net.URISyntaxException: Illegal character in scheme name at
index 0:
"hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.checkChar(URI.java:3012)
at java.net.URI$Parser.parse(URI.java:3028)
at java.net.URI.<init>(URI.java:753)
at org.apache.hadoop.fs.Path.initialize(Path.java:203)
Anything wrong with my configuration for TEZ?
Dr Mich Talebzadeh
LinkedIn
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABU
rV8Pw>
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUr
V8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
<http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908
.pdf>
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.
pdf
Author of the books "A Practitioner's Guide to Upgrading to Sybase ASE 15",
ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN
978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN:
978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume
one out shortly
<http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees accept
any responsibility.
RE: Hive on TEZ fails starting
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Apologies it starts OK and fails at reducer!
--------------------------------------------------------------------------------
Map 1 RUNNING 1 0 1 0 3 0
Reducer 2 INITED 1 0 0 1 0 0p: application_1452068056412_0006 dag:dag_1452068056412_0006_1
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
16/01/06 15:53:53 [main]: DEBUG rpc.DAGClientRPCImpl: GetVertexStatus via AM for app: application_1452068056412_0006 dag: dag_1452068056412_0006_1 vertex: Map 1
16/01/06 15:53:53 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (1353540671) connection to rhes564/50.140.197.217:56764 from hduser sending #152
16/01/06 15:53:53 [IPC Client (1353540671) connection to rhes564/50.140.197.217:56764 from hduser]: DEBUG ipc.Client: IPC Client (1353540671) connection to rhes564/50.140.197.217:56764 from hduser got value #152
16/01/06 15:53:53 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getVertexStatus took 10msfrom hduser]: DEBUG ipc.Client: IPC Client (1353540671) connection to rhes564/50.140.197.217:56764 from hduser got value #151
16/01/06 15:53:53 [main]: DEBUG rpc.DAGClientRPCImpl: GetVertexStatus via AM for app: application_1452068056412_0006 dag: dag_1452068056412_0006_1 vertex: Reducer 2
16/01/06 15:53:53 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (1353540671) connection to rhes564/50.140.197.217:56764 from hduser sending #153
16/01/06 15:53:53 [IPC Client (1353540671) connection to rhes564/50.140.197.217:56764 from hduser]: DEBUG ipc.Client: IPC Client (1353540671) connection to rhes564/50.140.197.217:56764 from hduser got value #153
16/01/06 15:53:53 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getVertexStatus took 2ms
Map 1 FAILED 1 0 0 1 4 0
Reducer 2 KILLED 1 0 0 1 0 0
--------------------------------------------------------------------------------
VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 13.31 s
--------------------------------------------------------------------------------
16/01/06 15:53:53 [main]: INFO SessionState: Map 1: 0(+0,-4)/1 Reducer 2: 0/1
Status: Failed
16/01/06 15:53:53 [main]: ERROR SessionState: Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1452068056412_0006_1_00, diagnostics=[Task failed, taskId=task_1452068056412_0006_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:157)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Mich Talebzadeh [mailto:mich@peridale.co.uk]
Sent: 06 January 2016 15:03
To: user@hive.apache.org
Subject: RE: Hive on TEZ fails starting
Not starting at all!
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
Sent: 06 January 2016 09:18
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Re: Hive on TEZ fails starting
Is the job starting and getting stuck in the reducer like you mentioned in the initial mail? or the job itself is not starting?
~Rajesh.B
On Wed, Jan 6, 2016 at 1:47 PM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Hi,
Thanks for your help. I downloaded and installed snappy libraries as they were missing.
Setting Hive execution engine to tez and doing a simple query. Hive is stuck
> set hive.execution.engine=tez;
16/01/06 08:20:22 [main]: DEBUG parse.VariableSubstitution: Substitution is on: tez
In debug mode I get
16/01/06 08:22:07 [IPC Client (2116259755 <tel:%282116259755> ) connection to localhost/127.0.0.1:8032 <http://127.0.0.1:8032> from hduser]: DEBUG ipc.Client: IPC Client (2116259755 <tel:%282116259755> ) connection to localhost/127.0.0.1:8032 <http://127.0.0.1:8032> from hduser got value #188
16/01/06 08:22:07 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/01/06 08:22:07 [LeaseRenewer:hduser@rhes564:9000]: DEBUG hdfs.LeaseRenewer: Lease renewer daemon for [] with renew id 1 executed
Which does not progress. Sounds like the same issue as I saw with Spark.
Can I do something like?
set tez.master=yarn-client;
Thanks
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com <ma...@gmail.com> ]
Sent: 05 January 2016 11:46
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Re: Hive on TEZ fails starting
Try ' beeline --hiveconf tez.task.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" --hiveconf tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" '. Please check if you have the lib*.so available in the native folder (or point it to the folder which contains the so files)
~Rajesh.B
On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Hi,
I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
Trying to use TEZ, I still get the same error
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.002 seconds)
0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
No rows affected (0.019 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1451986680090_0002)
INFO : Map 1: -/- Reducer 2: 0/1
INFO : Map 1: 0/1 Reducer 2: 0/1
INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed, taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com <ma...@gmail.com> ]
Sent: 05 January 2016 00:35
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Re: Hive on TEZ fails starting
By default it should add "LD_LIBRARY_PATH" in the container (ref: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358) . In case your snappy native libs are present elsewhere in the cluster deployment, you can override using "tez.task.launch.env" and "tez.am.launch.env" (refer: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412)
~Rajesh.B
--
~Rajesh.B
--
~Rajesh.B
RE: Hive on TEZ fails starting
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Not starting at all!
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
Sent: 06 January 2016 09:18
To: user@hive.apache.org
Subject: Re: Hive on TEZ fails starting
Is the job starting and getting stuck in the reducer like you mentioned in the initial mail? or the job itself is not starting?
~Rajesh.B
On Wed, Jan 6, 2016 at 1:47 PM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Hi,
Thanks for your help. I downloaded and installed snappy libraries as they were missing.
Setting Hive execution engine to tez and doing a simple query. Hive is stuck
> set hive.execution.engine=tez;
16/01/06 08:20:22 [main]: DEBUG parse.VariableSubstitution: Substitution is on: tez
In debug mode I get
16/01/06 08:22:07 [IPC Client (2116259755 <tel:%282116259755> ) connection to localhost/127.0.0.1:8032 <http://127.0.0.1:8032> from hduser]: DEBUG ipc.Client: IPC Client (2116259755 <tel:%282116259755> ) connection to localhost/127.0.0.1:8032 <http://127.0.0.1:8032> from hduser got value #188
16/01/06 08:22:07 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/01/06 08:22:07 [LeaseRenewer:hduser@rhes564:9000]: DEBUG hdfs.LeaseRenewer: Lease renewer daemon for [] with renew id 1 executed
Which does not progress. Sounds like the same issue as I saw with Spark.
Can I do something like?
set tez.master=yarn-client;
Thanks
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com <ma...@gmail.com> ]
Sent: 05 January 2016 11:46
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Re: Hive on TEZ fails starting
Try ' beeline --hiveconf tez.task.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" --hiveconf tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" '. Please check if you have the lib*.so available in the native folder (or point it to the folder which contains the so files)
~Rajesh.B
On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Hi,
I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
Trying to use TEZ, I still get the same error
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.002 seconds)
0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
No rows affected (0.019 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1451986680090_0002)
INFO : Map 1: -/- Reducer 2: 0/1
INFO : Map 1: 0/1 Reducer 2: 0/1
INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed, taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com <ma...@gmail.com> ]
Sent: 05 January 2016 00:35
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Re: Hive on TEZ fails starting
By default it should add "LD_LIBRARY_PATH" in the container (ref: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358) . In case your snappy native libs are present elsewhere in the cluster deployment, you can override using "tez.task.launch.env" and "tez.am.launch.env" (refer: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412)
~Rajesh.B
--
~Rajesh.B
--
~Rajesh.B
Re: Hive on TEZ fails starting
Posted by Rajesh Balamohan <ra...@gmail.com>.
Is the job starting and getting stuck in the reducer like you mentioned in
the initial mail? or the job itself is not starting?
~Rajesh.B
On Wed, Jan 6, 2016 at 1:47 PM, Mich Talebzadeh <mi...@peridale.co.uk> wrote:
> Hi,
>
>
>
> Thanks for your help. I downloaded and installed snappy libraries as they
> were missing.
>
>
>
> Setting Hive execution engine to tez and doing a simple query. Hive is
> stuck
>
>
>
> > set hive.execution.engine=tez;
>
> 16/01/06 08:20:22 [main]: DEBUG parse.VariableSubstitution: Substitution
> is on: tez
>
>
>
>
>
>
>
> In debug mode I get
>
>
>
> 16/01/06 08:22:07 [IPC Client (2116259755) connection to localhost/
> 127.0.0.1:8032 from hduser]: DEBUG ipc.Client: IPC Client (2116259755)
> connection to localhost/127.0.0.1:8032 from hduser got value #188
>
> 16/01/06 08:22:07 [main]: DEBUG ipc.ProtobufRpcEngine: Call:
> getApplicationReport took 1ms
>
> 16/01/06 08:22:07 [LeaseRenewer:hduser@rhes564:9000]: DEBUG
> hdfs.LeaseRenewer: Lease renewer daemon for [] with renew id 1 executed
>
>
>
>
>
> Which does not progress. Sounds like the same issue as I saw with Spark.
>
>
>
> Can I do something like?
>
>
>
> set tez.master=yarn-client;
>
>
>
>
>
> Thanks
>
>
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 11:46
>
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> Try ' beeline --hiveconf tez.task.launch.env="
> LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native"
> --hiveconf tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native"
> '. Please check if you have the lib*.so available in the native folder (or
> point it to the folder which contains the so files)
>
>
>
> ~Rajesh.B
>
>
>
>
>
> On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mi...@peridale.co.uk>
> wrote:
>
> Hi,
>
>
>
> I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
>
>
>
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
> export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
>
>
> Trying to use TEZ, I still get the same error
>
>
>
> 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
>
> No rows affected (0.002 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
>
> No rows affected (0.019 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
>
> INFO : Tez session hasn't been created yet. Opening session
>
> INFO :
>
>
>
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1451986680090_0002)
>
>
>
> INFO : Map 1: -/- Reducer 2: 0/1
>
> INFO : Map 1: 0/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
>
>
> ERROR : Vertex failed, vertexName=Map 1,
> vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed,
> taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
>
>
>
>
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 00:35
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> By default it should add "LD_LIBRARY_PATH" in the container (ref:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358)
> . In case your snappy native libs are present elsewhere in the cluster
> deployment, you can override using "tez.task.launch.env" and
> "tez.am.launch.env" (refer:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412
> )
>
>
>
> ~Rajesh.B
>
>
>
>
>
>
>
>
>
> --
>
> ~Rajesh.B
>
--
~Rajesh.B
RE: Hive on TEZ fails starting
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Hi,
Thanks for your help. I downloaded and installed snappy libraries as they were missing.
Setting Hive execution engine to tez and doing a simple query. Hive is stuck
> set hive.execution.engine=tez;
16/01/06 08:20:22 [main]: DEBUG parse.VariableSubstitution: Substitution is on: tez
In debug mode I get
16/01/06 08:22:07 [IPC Client (2116259755) connection to localhost/127.0.0.1:8032 from hduser]: DEBUG ipc.Client: IPC Client (2116259755) connection to localhost/127.0.0.1:8032 from hduser got value #188
16/01/06 08:22:07 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/01/06 08:22:07 [LeaseRenewer:hduser@rhes564:9000]: DEBUG hdfs.LeaseRenewer: Lease renewer daemon for [] with renew id 1 executed
Which does not progress. Sounds like the same issue as I saw with Spark.
Can I do something like?
set tez.master=yarn-client;
Thanks
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
Sent: 05 January 2016 11:46
To: user@hive.apache.org
Subject: Re: Hive on TEZ fails starting
Try ' beeline --hiveconf tez.task.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" --hiveconf tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" '. Please check if you have the lib*.so available in the native folder (or point it to the folder which contains the so files)
~Rajesh.B
On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Hi,
I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
Trying to use TEZ, I still get the same error
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.002 seconds)
0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
No rows affected (0.019 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1451986680090_0002)
INFO : Map 1: -/- Reducer 2: 0/1
INFO : Map 1: 0/1 Reducer 2: 0/1
INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed, taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com <ma...@gmail.com> ]
Sent: 05 January 2016 00:35
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Re: Hive on TEZ fails starting
By default it should add "LD_LIBRARY_PATH" in the container (ref: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358) . In case your snappy native libs are present elsewhere in the cluster deployment, you can override using "tez.task.launch.env" and "tez.am.launch.env" (refer: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412)
~Rajesh.B
--
~Rajesh.B
RE: Hive on TEZ fails starting
Posted by Artem Ervits <ar...@gmail.com>.
Check if you have conflicting java versions
On Jan 5, 2016 5:27 PM, "Mich Talebzadeh" <mi...@peridale.co.uk> wrote:
> Hi Rajesh,
>
>
>
> This is what I have under :$HADOOP_COMMON_HOME/lib/native
>
>
>
> cd $HADOOP_COMMON_HOME/lib/native
>
> hduser@rhes564::/home/hduser/hadoop-2.6.0/lib/native> ls -ltr
>
> total 4936
>
> -rwxr-xr-x 1 hduser hadoop 278622 Nov 13 2014 libhdfs.so.0.0.0
>
> -rw-r-xr-x 1 hduser hadoop 440498 Nov 13 2014 libhdfs.a
>
> -rw-r-xr-x 1 hduser hadoop 476666 Nov 13 2014 libhadooputils.a
>
> -rw-r-xr-x 1 hduser hadoop 1634592 Nov 13 2014 libhadooppipes.a
>
> -rwxr-xr-x 1 hduser hadoop 805999 Nov 13 2014 libhadoop.so.1.0.0
>
> -rw-r-xr-x 1 hduser hadoop 1380212 Nov 13 2014 libhadoop.a
>
> lrwxrwxrwx 1 hduser hadoop 16 Feb 8 2015 libhdfs.so ->
> libhdfs.so.0.0.0
>
> lrwxrwxrwx 1 hduser hadoop 18 Feb 8 2015 libhadoop.so ->
> libhadoop.so.1.0.0
>
>
>
>
>
> And If I do a search for snappy I get
>
>
>
> hduser@rhes564::/home/hduser/hadoop-2.6.0> find ./ -name '*snappy*'
>
> ./share/hadoop/common/lib/snappy-java-1.0.4.1.jar
>
> ./share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/snappy-java-1.0.4.1.jar
>
>
> ./share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/snappy-java-1.0.4.1.jar
>
> ./share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar
>
> ./share/hadoop/tools/lib/snappy-java-1.0.4.1.jar
>
>
>
>
>
> Thanks,
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 11:46
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> Try ' beeline --hiveconf tez.task.launch.env="
> LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native"
> --hiveconf tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native"
> '. Please check if you have the lib*.so available in the native folder (or
> point it to the folder which contains the so files)
>
>
>
> ~Rajesh.B
>
>
>
>
>
> On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mi...@peridale.co.uk>
> wrote:
>
> Hi,
>
>
>
> I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
>
>
>
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
> export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
>
>
> Trying to use TEZ, I still get the same error
>
>
>
> 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
>
> No rows affected (0.002 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
>
> No rows affected (0.019 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
>
> INFO : Tez session hasn't been created yet. Opening session
>
> INFO :
>
>
>
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1451986680090_0002)
>
>
>
> INFO : Map 1: -/- Reducer 2: 0/1
>
> INFO : Map 1: 0/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
>
>
> ERROR : Vertex failed, vertexName=Map 1,
> vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed,
> taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
>
>
>
>
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 00:35
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> By default it should add "LD_LIBRARY_PATH" in the container (ref:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358)
> . In case your snappy native libs are present elsewhere in the cluster
> deployment, you can override using "tez.task.launch.env" and
> "tez.am.launch.env" (refer:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412
> )
>
>
>
> ~Rajesh.B
>
>
>
>
>
>
>
>
>
> --
>
> ~Rajesh.B
>
RE: Hive on TEZ fails starting
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Hi Rajesh,
This is what I have under :$HADOOP_COMMON_HOME/lib/native
cd $HADOOP_COMMON_HOME/lib/native
hduser@rhes564::/home/hduser/hadoop-2.6.0/lib/native> ls -ltr
total 4936
-rwxr-xr-x 1 hduser hadoop 278622 Nov 13 2014 libhdfs.so.0.0.0
-rw-r-xr-x 1 hduser hadoop 440498 Nov 13 2014 libhdfs.a
-rw-r-xr-x 1 hduser hadoop 476666 Nov 13 2014 libhadooputils.a
-rw-r-xr-x 1 hduser hadoop 1634592 Nov 13 2014 libhadooppipes.a
-rwxr-xr-x 1 hduser hadoop 805999 Nov 13 2014 libhadoop.so.1.0.0
-rw-r-xr-x 1 hduser hadoop 1380212 Nov 13 2014 libhadoop.a
lrwxrwxrwx 1 hduser hadoop 16 Feb 8 2015 libhdfs.so -> libhdfs.so.0.0.0
lrwxrwxrwx 1 hduser hadoop 18 Feb 8 2015 libhadoop.so -> libhadoop.so.1.0.0
And If I do a search for snappy I get
hduser@rhes564::/home/hduser/hadoop-2.6.0> find ./ -name '*snappy*'
./share/hadoop/common/lib/snappy-java-1.0.4.1.jar
./share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/snappy-java-1.0.4.1.jar
./share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/snappy-java-1.0.4.1.jar
./share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar
./share/hadoop/tools/lib/snappy-java-1.0.4.1.jar
Thanks,
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
Sent: 05 January 2016 11:46
To: user@hive.apache.org
Subject: Re: Hive on TEZ fails starting
Try ' beeline --hiveconf tez.task.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" --hiveconf tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" '. Please check if you have the lib*.so available in the native folder (or point it to the folder which contains the so files)
~Rajesh.B
On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Hi,
I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
Trying to use TEZ, I still get the same error
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.002 seconds)
0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
No rows affected (0.019 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1451986680090_0002)
INFO : Map 1: -/- Reducer 2: 0/1
INFO : Map 1: 0/1 Reducer 2: 0/1
INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed, taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com <ma...@gmail.com> ]
Sent: 05 January 2016 00:35
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Re: Hive on TEZ fails starting
By default it should add "LD_LIBRARY_PATH" in the container (ref: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358) . In case your snappy native libs are present elsewhere in the cluster deployment, you can override using "tez.task.launch.env" and "tez.am.launch.env" (refer: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412)
~Rajesh.B
--
~Rajesh.B
Re: Hive on TEZ fails starting
Posted by Rajesh Balamohan <ra...@gmail.com>.
Try ' beeline --hiveconf tez.task.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_
PATH:$HADOOP_COMMON_HOME/lib/native" --hiveconf tez.am.launch.env="
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" '. Please
check if you have the lib*.so available in the native folder (or point it
to the folder which contains the so files)
~Rajesh.B
On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mi...@peridale.co.uk> wrote:
> Hi,
>
>
>
> I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
>
>
>
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
> export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
>
>
> Trying to use TEZ, I still get the same error
>
>
>
> 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
>
> No rows affected (0.002 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
>
> No rows affected (0.019 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
>
> INFO : Tez session hasn't been created yet. Opening session
>
> INFO :
>
>
>
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1451986680090_0002)
>
>
>
> INFO : Map 1: -/- Reducer 2: 0/1
>
> INFO : Map 1: 0/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
>
>
> ERROR : Vertex failed, vertexName=Map 1,
> vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed,
> taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
>
>
>
>
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 00:35
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> By default it should add "LD_LIBRARY_PATH" in the container (ref:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358)
> . In case your snappy native libs are present elsewhere in the cluster
> deployment, you can override using "tez.task.launch.env" and
> "tez.am.launch.env" (refer:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412
> )
>
>
>
> ~Rajesh.B
>
>
>
>
>
--
~Rajesh.B
RE: Hive on TEZ fails starting
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Hi,
I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
Trying to use TEZ, I still get the same error
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.002 seconds)
0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
No rows affected (0.019 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1451986680090_0002)
INFO : Map 1: -/- Reducer 2: 0/1
INFO : Map 1: 0/1 Reducer 2: 0/1
INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed, taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
Sent: 05 January 2016 00:35
To: user@hive.apache.org
Subject: Re: Hive on TEZ fails starting
By default it should add "LD_LIBRARY_PATH" in the container (ref: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358) . In case your snappy native libs are present elsewhere in the cluster deployment, you can override using "tez.task.launch.env" and "tez.am.launch.env" (refer: https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412)
~Rajesh.B
Re: Hive on TEZ fails starting
Posted by Rajesh Balamohan <ra...@gmail.com>.
By default it should add "LD_LIBRARY_PATH" in the container (ref:
https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358)
. In case your snappy native libs are present elsewhere in the cluster
deployment, you can override using "tez.task.launch.env" and
"tez.am.launch.env" (refer:
https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412
)
~Rajesh.B
On Tue, Jan 5, 2016 at 5:45 AM, Mich Talebzadeh <mi...@peridale.co.uk> wrote:
> Thanks Rajesh.
>
>
>
> Done that and progressed
>
>
>
> Got another error L
>
>
>
> 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
>
> No rows affected (0.003 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
>
> INFO : Tez session hasn't been created yet. Opening session
>
> INFO :
>
>
>
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1451778329897_0084)
>
>
>
> INFO : Map 1: -/- Reducer 2: 0/1
>
> INFO : Map 1: 0/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
> INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
>
> ERROR : Status: Failed
>
> ERROR : Vertex failed, vertexName=Map 1,
> vertexId=vertex_1451778329897_0084_1_00, diagnostics=[Task failed,
> taskId=task_1451778329897_0084_1_00_000000, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 00:07
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> Can you try removing double-quotes for "tez.lib.uris" in tez-site.xml (i.e
> just use
> hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz)?
>
>
>
> ~Rajesh.B
>
>
>
>
>
>
>
> On Tue, Jan 5, 2016 at 5:30 AM, Mich Talebzadeh <mi...@peridale.co.uk>
> wrote:
>
> Hi,
>
>
>
> Trying to run Hive on TEZ for the first time. Getting the error below
>
>
>
>
>
>
>
> 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
>
> No rows affected (0.001 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
>
> No rows affected (0.018 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
>
> INFO : Tez session hasn't been created yet. Opening session
>
> ERROR : Failed to execute tez graph.
>
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal
> character in scheme name at index 0:
> "hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
>
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
>
> at org.apache.hadoop.fs.Path.<init>(Path.java:172)
>
> at
> org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:964)
>
> at
> org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:228)
>
> at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:203)
>
> at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:883)
>
> at org.apache.tez.client.TezClient.start(TezClient.java:365)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:184)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:239)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:137)
>
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
>
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
>
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
>
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
>
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
>
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
>
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:724)
>
> Caused by: java.net.URISyntaxException: Illegal character in scheme name
> at index 0:
> "hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
>
> at java.net.URI$Parser.fail(URI.java:2829)
>
> at java.net.URI$Parser.checkChars(URI.java:3002)
>
> at java.net.URI$Parser.checkChar(URI.java:3012)
>
> at java.net.URI$Parser.parse(URI.java:3028)
>
> at java.net.URI.<init>(URI.java:753)
>
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
>
>
>
>
>
> Anything wrong with my configuration for TEZ?
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
>
>
>
>
> --
>
> ~Rajesh.B
>
--
~Rajesh.B
RE: Hive on TEZ fails starting
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Thanks Rajesh.
Done that and progressed
Got another error :(
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.003 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_1451778329897_0084)
INFO : Map 1: -/- Reducer 2: 0/1
INFO : Map 1: 0/1 Reducer 2: 0/1
INFO : Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1
ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1451778329897_0084_1_00, diagnostics=[Task failed, taskId=task_1451778329897_0084_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
Sent: 05 January 2016 00:07
To: user@hive.apache.org
Subject: Re: Hive on TEZ fails starting
Can you try removing double-quotes for "tez.lib.uris" in tez-site.xml (i.e just use hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz)?
~Rajesh.B
On Tue, Jan 5, 2016 at 5:30 AM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Hi,
Trying to run Hive on TEZ for the first time. Getting the error below
0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
No rows affected (0.001 seconds)
0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
No rows affected (0.018 seconds)
0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
INFO : Tez session hasn't been created yet. Opening session
ERROR : Failed to execute tez graph.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in scheme name at index 0: "hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
at org.apache.hadoop.fs.Path.initialize(Path.java:206)
at org.apache.hadoop.fs.Path.<init>(Path.java:172)
at org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:964)
at org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:228)
at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:203)
at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:883)
at org.apache.tez.client.TezClient.start(TezClient.java:365)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:184)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:239)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0: "hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.checkChar(URI.java:3012)
at java.net.URI$Parser.parse(URI.java:3028)
at java.net.URI.<init>(URI.java:753)
at org.apache.hadoop.fs.Path.initialize(Path.java:203)
Anything wrong with my configuration for TEZ?
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
--
~Rajesh.B
Re: Hive on TEZ fails starting
Posted by Rajesh Balamohan <ra...@gmail.com>.
Can you try removing double-quotes for "tez.lib.uris" in tez-site.xml (i.e
just use hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-
SNAPSHOT.tar.gz)?
~Rajesh.B
On Tue, Jan 5, 2016 at 5:30 AM, Mich Talebzadeh <mi...@peridale.co.uk> wrote:
> Hi,
>
>
>
> Trying to run Hive on TEZ for the first time. Getting the error below
>
>
>
>
>
>
>
> 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
>
> No rows affected (0.001 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
>
> No rows affected (0.018 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
>
> INFO : Tez session hasn't been created yet. Opening session
>
> ERROR : Failed to execute tez graph.
>
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal
> character in scheme name at index 0:
> "hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
>
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
>
> at org.apache.hadoop.fs.Path.<init>(Path.java:172)
>
> at
> org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:964)
>
> at
> org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:228)
>
> at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:203)
>
> at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:883)
>
> at org.apache.tez.client.TezClient.start(TezClient.java:365)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:184)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:239)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:137)
>
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
>
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
>
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
>
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
>
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
>
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
>
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:724)
>
> Caused by: java.net.URISyntaxException: Illegal character in scheme name
> at index 0:
> "hdfs://rhes564:9000/apps/tez-0.7.1-SNAPSHOT/tez-0.7.1-SNAPSHOT.tar.gz%22
>
> at java.net.URI$Parser.fail(URI.java:2829)
>
> at java.net.URI$Parser.checkChars(URI.java:3002)
>
> at java.net.URI$Parser.checkChar(URI.java:3012)
>
> at java.net.URI$Parser.parse(URI.java:3028)
>
> at java.net.URI.<init>(URI.java:753)
>
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
>
>
>
>
>
> Anything wrong with my configuration for TEZ?
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
--
~Rajesh.B