You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by "Meier, Alexander" <Al...@t-systems-dmc.com> on 2017/05/19 14:20:26 UTC
Hive interpreter Error as soon as Hive query uses MapRed
Hi list
I’m trying to get a Hive interpreter correctly running on a CDH 5.7 Cluster with Spark 1.6. Simple queries are running fine, but as soon as a query needs a MapRed tasks in order to complete, the query fails with:
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:580)
at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:95)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
etc…
I’ve got the interpreter set up as follows:
Properties
name value
default.driver org.apache.hive.jdbc.HiveDriver
default.url jdbc:hive2://[hostname]:10000
hive.driver org.apache.hive.jdbc.HiveDriver
hive.url jdbc:hive2://[hostname]:10000
zeppelin.interpreter.localRepo /opt/zeppelin/local-repo/2CJ4XM2Z4
Dependencies
artifact
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
/opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar
Unfortunately I haven’t found any help googling around… anyone here with some helpful input?
Best regards and many thanks in advance,
Alex
Re: Hive interpreter Error as soon as Hive query uses MapRed
Posted by Ben Vogan <be...@shopkick.com>.
I am running CDH 5.7 and Spark 1.6 as well and hive is working for me with
the following configuration:
Properties
namevalue
common.max_count 1000
default.driver org.apache.hive.jdbc.HiveDriver
default.password
default.url jdbc:hive2://hdfs004:10000
default.user hive
zeppelin.interpreter.localRepo
/services/zeppelin/zeppelin-0.7.1/local-repo/2CECB8FBV
zeppelin.jdbc.auth.type
zeppelin.jdbc.concurrent.max_connection 10
zeppelin.jdbc.concurrent.use true
zeppelin.jdbc.keytab.location
zeppelin.jdbc.principal
Dependencies
artifactexclude
org.apache.hive:hive-jdbc:0.14.0
org.apache.hadoop:hadoop-common:2.6.0
I admit to not having spent time figuring out whether there are any edge
cases that are broken because I am using the open source version of the
odbc driver vs using the cloudera jars. However, it definitely returns
results from complex select queries and has no issues with DDL statements
that I've tried.
Good luck!
--Ben
On Fri, May 19, 2017 at 12:10 PM, Meier, Alexander <
Alexander.Meier@t-systems-dmc.com> wrote:
> Yes, the script (i.e. The select statement) runs fine in hive cli, hue and
> also in spark sql ( spark sql also in zeppelin).
> Just not when using the hive interpreter in zeppelin.
>
>
>
> Sent from my iPhone
>
> Am 19.05.2017 um 19:35 schrieb Jongyoul Lee <jo...@gmail.com>:
>
> Can you check your script works in native hive environment?
>
> On Fri, May 19, 2017 at 10:20 AM, Meier, Alexander <
> Alexander.Meier@t-systems-dmc.com> wrote:
>
>> Hi list
>>
>> I’m trying to get a Hive interpreter correctly running on a CDH 5.7
>> Cluster with Spark 1.6. Simple queries are running fine, but as soon as a
>> query needs a MapRed tasks in order to complete, the query fails with:
>>
>> java.sql.SQLException: Error while processing statement: FAILED:
>> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec
>> .mr.MapRedTask
>> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.
>> java:279)
>> at org.apache.commons.dbcp2.DelegatingStatement.execute(Delegat
>> ingStatement.java:291)
>> at org.apache.commons.dbcp2.DelegatingStatement.execute(Delegat
>> ingStatement.java:291)
>> at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInte
>> rpreter.java:580)
>> at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInter
>> preter.java:692)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.
>> interpret(LazyOpenInterpreter.java:95)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
>> er$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
>> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOSchedu
>> ler.java:139)
>> at java.util.concurrent.Executors$RunnableAdapter.call(
>> Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>> etc…
>>
>> I’ve got the interpreter set up as follows:
>>
>> Properties
>> name value
>> default.driver
>> org.apache.hive.jdbc.HiveDriver
>> default.url
>> jdbc:hive2://[hostname]:10000
>> hive.driver
>> org.apache.hive.jdbc.HiveDriver
>> hive.url
>> jdbc:hive2://[hostname]:10000
>> zeppelin.interpreter.localRepo /opt/zeppelin/local-repo/2CJ4XM2Z4
>>
>> Dependencies
>> artifact
>> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
>> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
>> /opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
>> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
>> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar
>>
>>
>> Unfortunately I haven’t found any help googling around… anyone here with
>> some helpful input?
>>
>> Best regards and many thanks in advance,
>> Alex
>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
--
*BENJAMIN VOGAN* | Data Platform Team Lead
<http://www.shopkick.com/>
<https://www.facebook.com/shopkick> <https://www.instagram.com/shopkick/>
<https://www.pinterest.com/shopkick/> <https://twitter.com/shopkickbiz>
<https://www.linkedin.com/company-beta/831240/?pathWildcard=831240>
Re: Hive interpreter Error as soon as Hive query uses MapRed
Posted by "Meier, Alexander" <Al...@t-systems-dmc.com>.
Yes, the script (i.e. The select statement) runs fine in hive cli, hue and also in spark sql ( spark sql also in zeppelin).
Just not when using the hive interpreter in zeppelin.
Sent from my iPhone
Am 19.05.2017 um 19:35 schrieb Jongyoul Lee <jo...@gmail.com>>:
Can you check your script works in native hive environment?
On Fri, May 19, 2017 at 10:20 AM, Meier, Alexander <Al...@t-systems-dmc.com>> wrote:
Hi list
I'm trying to get a Hive interpreter correctly running on a CDH 5.7 Cluster with Spark 1.6. Simple queries are running fine, but as soon as a query needs a MapRed tasks in order to complete, the query fails with:
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:580)
at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:95)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
etc...
I've got the interpreter set up as follows:
Properties
name value
default.driver org.apache.hive.jdbc.HiveDriver
default.url jdbc:hive2://[hostname]:10000
hive.driver org.apache.hive.jdbc.HiveDriver
hive.url jdbc:hive2://[hostname]:10000
zeppelin.interpreter.localRepo /opt/zeppelin/local-repo/2CJ4XM2Z4
Dependencies
artifact
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
/opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar
Unfortunately I haven't found any help googling around... anyone here with some helpful input?
Best regards and many thanks in advance,
Alex
--
???, Jongyoul Lee, ???
http://madeng.net
Re: Hive interpreter Error as soon as Hive query uses MapRed
Posted by Jongyoul Lee <jo...@gmail.com>.
Can you check your script works in native hive environment?
On Fri, May 19, 2017 at 10:20 AM, Meier, Alexander <
Alexander.Meier@t-systems-dmc.com> wrote:
> Hi list
>
> I’m trying to get a Hive interpreter correctly running on a CDH 5.7
> Cluster with Spark 1.6. Simple queries are running fine, but as soon as a
> query needs a MapRed tasks in order to complete, the query fails with:
>
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at org.apache.hive.jdbc.HiveStatement.execute(
> HiveStatement.java:279)
> at org.apache.commons.dbcp2.DelegatingStatement.execute(
> DelegatingStatement.java:291)
> at org.apache.commons.dbcp2.DelegatingStatement.execute(
> DelegatingStatement.java:291)
> at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(
> JDBCInterpreter.java:580)
> at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(
> JDBCInterpreter.java:692)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(
> LazyOpenInterpreter.java:95)
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$
> InterpretJob.jobRun(RemoteInterpreterServer.java:490)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(
> FIFOScheduler.java:139)
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> etc…
>
> I’ve got the interpreter set up as follows:
>
> Properties
> name value
> default.driver org.apache.hive.jdbc.
> HiveDriver
> default.url
> jdbc:hive2://[hostname]:10000
> hive.driver org.apache.hive.jdbc.
> HiveDriver
> hive.url
> jdbc:hive2://[hostname]:10000
> zeppelin.interpreter.localRepo /opt/zeppelin/local-repo/2CJ4XM2Z4
>
> Dependencies
> artifact
> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
> /opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
> /opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar
>
>
> Unfortunately I haven’t found any help googling around… anyone here with
> some helpful input?
>
> Best regards and many thanks in advance,
> Alex
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net