You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Rajesh Narayanan <ra...@oracle.com> on 2017/09/11 12:36:50 UTC
Hive LLAP service is not starting
Hi All,
I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to execute hive query in yarn as TEZ type.
When I start hive LLAP service -./hive --service llap --name @llap --instances 1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB"
Getting below exception. Could you please suggest what missing here.
Failed: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:605)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:113)
Caused by: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.hadoop.hive.common.CompressionUtils.unTar(CompressionUtils.java:152)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:361)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:348)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
INFO cli.LlapServiceDriver: LLAP service driver finished
Thanks
Rajesh
Re: Hive LLAP service is not starting
Posted by dan young <da...@gmail.com>.
Hello Rajesh,
Here's what we use on Hive 2.3 and Hadoop 2.7.4
export PATH=$PATH:/usr/lib/slider/bin
export HADOOP_CONF_DIR=/usr/lib/hadoop/conf
export JAVA_HOME=/usr/java/default
export HADOOP_HOME=/usr/lib/hadoop
/usr/lib/hive/bin/hive --service llap --name llaptest --instances 2 --size
6144m --cache 512m --slider-am-container-mb 2048 --queue default
--executors 4 --xmx 5120m --loglevel INFO --auxjars
<add-your-aux-jars-here> --args " -XX:+AlwaysPreTouch -XX:+UseG1GC
-XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts
-XX:InitiatingHeapOccupancyPercent=40 -XX:G1ReservePercent=20
-XX:MaxGCPauseMillis=200 -XX:MetaspaceSize=512m" --slider-appconfig-global
site.global.app_user=hadoop
this will create a run.sh file in the folder that you are running the
command from. The run the run.sh, which will submit to YARN.
./llap-slider-24Aug2017/run.sh
you can check the status either thru the YARN UI and/or slider command....
Regards,
Dano
On Tue, Sep 12, 2017 at 3:15 AM Rajesh Narayanan <
rajesh.r.narayanan@oracle.com> wrote:
> Hi Premal
>
> Thanks for the reply. I am following below document. Can you please
> provide any document or steps to start LLAP through slide
>
>
> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/intro_new_book.html
>
> Regards
>
> Rajesh
>
> *From:* Premal Shah [mailto:premal.j.shah@gmail.com]
> *Sent:* Monday, September 11, 2017 11:29 PM
> *To:* user@hive.apache.org
> *Subject:* Re: Hive LLAP service is not starting
>
>
>
> Hi Rajesh,
>
> I thought the recommended way to start/stop LLAP was to use Slider
>
> this is what we do to start
>
>
>
> slider create llap --resources $BASEDIR/resources.json --template
> $BASEDIR/appConfig.json
>
>
>
> and to stop before you can start again
>
>
>
> slider stop llap
>
> slider destroy llap --force || slider destroy llap
>
>
>
>
>
> On Mon, Sep 11, 2017 at 5:36 AM, Rajesh Narayanan <
> rajesh.r.narayanan@oracle.com> wrote:
>
> Hi All,
>
> I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to
> execute hive query in yarn as TEZ type.
>
> When I start hive LLAP service -./hive --service llap --name @llap
> --instances 1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG
> --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB"
>
> Getting below exception. Could you please suggest what missing here.
>
>
>
> Failed: java.io.FileNotFoundException:
> /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
> java.util.concurrent.ExecutionException: java.io.FileNotFoundException:
> /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:605)
> at
> org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:113)
> Caused by: java.io.FileNotFoundException:
> /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
> at
> org.apache.hadoop.hive.common.CompressionUtils.unTar(CompressionUtils.java:152)
> at
> org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:361)
> at
> org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:348)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> INFO cli.LlapServiceDriver: LLAP service driver finished
>
>
>
> Thanks
>
> Rajesh
>
>
>
>
>
> --
>
> Regards,
> Premal Shah.
>
RE: Hive LLAP service is not starting
Posted by Rajesh Narayanan <ra...@oracle.com>.
Hi Premal
Thanks for the reply. I am following below document. Can you please provide any document or steps to start LLAP through slide
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/intro_new_book.html
Regards
Rajesh
From: Premal Shah [mailto:premal.j.shah@gmail.com]
Sent: Monday, September 11, 2017 11:29 PM
To: user@hive.apache.org
Subject: Re: Hive LLAP service is not starting
Hi Rajesh,
I thought the recommended way to start/stop LLAP was to use Slider
this is what we do to start
slider create llap --resources $BASEDIR/resources.json --template $BASEDIR/appConfig.json
and to stop before you can start again
slider stop llap
slider destroy llap --force || slider destroy llap
On Mon, Sep 11, 2017 at 5:36 AM, Rajesh Narayanan <HYPERLINK "mailto:rajesh.r.narayanan@oracle.com" \nrajesh.r.narayanan@oracle.com> wrote:
Hi All,
I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to execute hive query in yarn as TEZ type.
When I start hive LLAP service -./hive --service llap --name @llap --instances 1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB"
Getting below exception. Could you please suggest what missing here.
Failed: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:605)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:113)
Caused by: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.hadoop.hive.common.CompressionUtils.unTar(CompressionUtils.java:152)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:361)
at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:348)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
INFO cli.LlapServiceDriver: LLAP service driver finished
Thanks
Rajesh
--
Regards,
Premal Shah.
Re: Hive LLAP service is not starting
Posted by Premal Shah <pr...@gmail.com>.
Hi Rajesh,
I thought the recommended way to start/stop LLAP was to use Slider
this is what we do to start
slider create llap --resources $BASEDIR/resources.json --template
$BASEDIR/appConfig.json
and to stop before you can start again
slider stop llap
slider destroy llap --force || slider destroy llap
On Mon, Sep 11, 2017 at 5:36 AM, Rajesh Narayanan <
rajesh.r.narayanan@oracle.com> wrote:
> Hi All,
>
> I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to
> execute hive query in yarn as TEZ type.
>
> When I start hive LLAP service -./hive --service llap --name @llap
> --instances 1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG
> --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB"
>
> Getting below exception. Could you please suggest what missing here.
>
>
>
> Failed: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz
> (Is a directory)
> java.util.concurrent.ExecutionException: java.io.FileNotFoundException:
> /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(
> LlapServiceDriver.java:605)
> at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(
> LlapServiceDriver.java:113)
> Caused by: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz
> (Is a directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
> at org.apache.hadoop.hive.common.CompressionUtils.unTar(
> CompressionUtils.java:152)
> at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(
> LlapServiceDriver.java:361)
> at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(
> LlapServiceDriver.java:348)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> INFO cli.LlapServiceDriver: LLAP service driver finished
>
>
>
> Thanks
>
> Rajesh
>
--
Regards,
Premal Shah.
Re: Hive LLAP service is not starting
Posted by Gopal Vijayaraghavan <go...@apache.org>.
> java.util.concurrent.ExecutionException: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
LLAP expects to find a tarball where tez.lib.uris is - looks like you've got a directory?
Cheers,
Gopal