You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Mistry, Jigar" <ji...@amazon.com> on 2017/02/16 20:04:37 UTC

Hive LLAP daemon failing with Tez

Hello,

I am testing out Hive’s LLAP functionality. My test environment is as follows:

-          Hadoop 2.7.3

-          Hive 2.1.1

-          Tez 0.8.4

-          Zookeeper 3.4.8

-          Slider 0.91

I am able to get the LLAP daemon running successfully and can see it in the RM UI. But when I start the Hive CLI and run a SELECT COUNT(*) query, the Tez DAG fails. Following is the stack trace I found inside the container logs:

[ERROR] [main] |app.DAGAppMaster|: Error starting DAGAppMaster
java.lang.NumberFormatException: null
                at java.lang.Integer.parseInt(Integer.java:542)
                at java.lang.Integer.parseInt(Integer.java:615)
                at org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl$DynamicServiceInstance.getResource(LlapZookeeperRegistryImpl.java:474)
                at org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeInfo.<init>(LlapTaskSchedulerService.java:1492)
                at org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.start(LlapTaskSchedulerService.java:319)
                at org.apache.tez.dag.app.ServicePluginLifecycleAbstractService.serviceStart(ServicePluginLifecycleAbstractService.java:41)
                at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
                at org.apache.tez.dag.app.rm.TaskSchedulerManager.serviceStart(TaskSchedulerManager.java:612)
                at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
                at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1827)
                at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1848)

Can someone help me figuring out what is going wrong here?

Thanks,
Jigar Mistry

Re: Hive LLAP daemon failing with Tez

Posted by Damien Carol <da...@gmail.com>.
For record,

We had the same issue in my company. We use Ambari and HDP. Ambari in HDP
2.5.3 doesn't provide well some configuration keys.

My advice for HDP users: checks the xml configuration files which Ambari
generate.

Regards,
Damien

2017-02-16 22:32 GMT+01:00 Mistry, Jigar <ji...@amazon.com>:

> Hi Gopal,
>
> Thanks for your response. I was indeed a configuration issue. I was able
> to make it work by using something like this:
>
> hive --service llap --name llap_test\
>  --instances 1\
>  --cache 2000m\
>  --executors 10\
>  --iothreads 10\
>  --size 5000m\
>  --xmx 2000m\
>  --loglevel WARN\
>  --args "-XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA  -XX:-ResizePLAB"\
>  --javaHome $JAVA_HOME
>
> Thanks again for pointing me in the correct direction.
>
> Regards,
> Jigar
>
> On 2/16/17, 12:23 PM, "Gopal Vijayaraghavan" <gopal@hortonworks.com on
> behalf of gopalv@apache.org> wrote:
>
>
>     >                at org.apache.hadoop.hive.llap.registry.impl.
> LlapZookeeperRegistryImpl$DynamicServiceInstance.getResource(
> LlapZookeeperRegistryImpl.java:474)
>
>     Most likely incorrect configuration.
>
>     hive.llap.daemon.num.executors
>     or
>     hive.llap.daemon.memory.per.instance.mb
>
>     Are you using something like this?
>
>     https://github.com/t3rmin4t0r/tez-autobuild/blob/llap/slider-gen.sh
>
>     Cheers,
>     Gopal
>
>
>
>
>
>

Re: Hive LLAP daemon failing with Tez

Posted by "Mistry, Jigar" <ji...@amazon.com>.
Hi Gopal,

Thanks for your response. I was indeed a configuration issue. I was able to make it work by using something like this:

hive --service llap --name llap_test\
 --instances 1\
 --cache 2000m\
 --executors 10\
 --iothreads 10\
 --size 5000m\
 --xmx 2000m\
 --loglevel WARN\
 --args "-XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA  -XX:-ResizePLAB"\
 --javaHome $JAVA_HOME

Thanks again for pointing me in the correct direction.

Regards,
Jigar

On 2/16/17, 12:23 PM, "Gopal Vijayaraghavan" <gopal@hortonworks.com on behalf of gopalv@apache.org> wrote:

    
    >                at org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl$DynamicServiceInstance.getResource(LlapZookeeperRegistryImpl.java:474)
    
    Most likely incorrect configuration.
    
    hive.llap.daemon.num.executors
    or
    hive.llap.daemon.memory.per.instance.mb
    
    Are you using something like this?
    
    https://github.com/t3rmin4t0r/tez-autobuild/blob/llap/slider-gen.sh
    
    Cheers,
    Gopal

Re: Hive LLAP daemon failing with Tez

Posted by Gopal Vijayaraghavan <go...@apache.org>.
>                at org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl$DynamicServiceInstance.getResource(LlapZookeeperRegistryImpl.java:474)

Most likely incorrect configuration.

hive.llap.daemon.num.executors
or
hive.llap.daemon.memory.per.instance.mb

Are you using something like this?

https://github.com/t3rmin4t0r/tez-autobuild/blob/llap/slider-gen.sh

Cheers,
Gopal