You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by 김동원 <ea...@gmail.com> on 2016/12/26 05:07:37 UTC
user@tez.apache.org
I want my IntelliJ to attach to the running LLAP damons by adding the following JVM option (for a debugging purpose):
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
Where is the right place to append the line?
My previous attempt was as follows :
- Before executing “hive --service llap …”, I modified ${hive-binary-dist}/scripts/llap/bin/llap-daemon-env.sh to modify LLAP_DAEMON_OPTS.
- What I found, however, is that ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh refers to ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh, not ${LLAP_DAEMON_BIN_HOME}
- I also tried to locate llap-daemon-env.sh under ${hive-binary-dost}/conf but found that LlapServiceDriver doesn’t copy that file under LLAP_DAEMON_CONF_DIR.
- I had to modify ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh to load llap-daemon-env.sh properly by making it refer to ${LLAP_DAEMON_BIN_HOME}/llap-daemon-env.sh instead of ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh.
- eastcircle
Re: user@tez.apache.org
Posted by Rajesh Balamohan <rb...@apache.org>.
Very nice post. Thanks for sharing.
btw, tez-autobuild is created and maintained by Gopal. :). I contribute
minor patches to it.
~Rajesh.B
On Mon, Dec 26, 2016 at 11:17 AM, 김동원 <ea...@gmail.com> wrote:
> Thanks a lot!
>
> I’m also aware of tez-autobuild from you.
> It works well in my cluster but this time I want to install
> hive/slider/tez by myself without tez-autobuild.
> Setting up and running LLAP are really tricky and I wrote a separate post
> in my blog to introduce my experience of setting up llap:
> http://eastcirclek.blogspot.com/2016/10/how-to-
> start-hive-llap-functionality.html.
>
>
> 2016. 12. 26., 오후 2:34, Rajesh Balamohan <ra...@gmail.com> 작성:
>
> Much easier option is to make use of https://github.com/
> t3rmin4t0r/tez-autobuild (edit/set args in slider-gen.sh).
>
> ~Rajesh.B
>
> On Mon, Dec 26, 2016 at 11:02 AM, Rajesh Balamohan <rb...@apache.org>
> wrote:
>
>> Here is an example:
>>
>> hive --service llap --instances 1 --args "-XX:+UseG1GC
>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000"
>> --cache 48000m --executors 8 --iothreads 8 --size 180000m --xmx 128000m
>> --loglevel INFO --javaHome /usr/jdk64/jdk1.8.0_77/
>>
>> This would generate a "run.sh" script which can be run to start LLAP.
>> This would open up 8000 port that you have mentioned in the machine where
>> LLAP daemon is started.
>>
>> ~Rajesh.B
>>
>> On Mon, Dec 26, 2016 at 10:37 AM, 김동원 <ea...@gmail.com> wrote:
>>
>>> I want my IntelliJ to attach to the running LLAP damons by adding the
>>> following JVM option (for a debugging purpose):
>>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
>>>
>>> Where is the right place to append the line?
>>>
>>> My previous attempt was as follows :
>>> - Before executing “hive --service llap …”, I modified
>>> ${hive-binary-dist}/scripts/llap/bin/llap-daemon-env.sh to modify
>>> LLAP_DAEMON_OPTS.
>>> - What I found, however, is that ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh
>>> refers to ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh, not
>>> ${LLAP_DAEMON_BIN_HOME}
>>> - I also tried to locate llap-daemon-env.sh under
>>> ${hive-binary-dost}/conf but found that LlapServiceDriver doesn’t copy that
>>> file under LLAP_DAEMON_CONF_DIR.
>>> - I had to modify ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh to
>>> load llap-daemon-env.sh properly by making it refer to
>>> ${LLAP_DAEMON_BIN_HOME}/llap-daemon-env.sh instead of
>>> ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh.
>>>
>>> - eastcircle
>>
>>
>>
>
>
> --
> ~Rajesh.B
>
>
>
Re: user@tez.apache.org
Posted by 김동원 <ea...@gmail.com>.
Thanks a lot!
I’m also aware of tez-autobuild from you.
It works well in my cluster but this time I want to install hive/slider/tez by myself without tez-autobuild.
Setting up and running LLAP are really tricky and I wrote a separate post in my blog to introduce my experience of setting up llap: http://eastcirclek.blogspot.com/2016/10/how-to-start-hive-llap-functionality.html <http://eastcirclek.blogspot.com/2016/10/how-to-start-hive-llap-functionality.html>.
> 2016. 12. 26., 오후 2:34, Rajesh Balamohan <ra...@gmail.com> 작성:
>
> Much easier option is to make use of https://github.com/t3rmin4t0r/tez-autobuild <https://github.com/t3rmin4t0r/tez-autobuild> (edit/set args in slider-gen.sh).
>
> ~Rajesh.B
>
> On Mon, Dec 26, 2016 at 11:02 AM, Rajesh Balamohan <rbalamohan@apache.org <ma...@apache.org>> wrote:
> Here is an example:
>
> hive --service llap --instances 1 --args "-XX:+UseG1GC -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000" --cache 48000m --executors 8 --iothreads 8 --size 180000m --xmx 128000m --loglevel INFO --javaHome /usr/jdk64/jdk1.8.0_77/
>
> This would generate a "run.sh" script which can be run to start LLAP. This would open up 8000 port that you have mentioned in the machine where LLAP daemon is started.
>
> ~Rajesh.B
>
> On Mon, Dec 26, 2016 at 10:37 AM, 김동원 <eastcirclek@gmail.com <ma...@gmail.com>> wrote:
> I want my IntelliJ to attach to the running LLAP damons by adding the following JVM option (for a debugging purpose):
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
>
> Where is the right place to append the line?
>
> My previous attempt was as follows :
> - Before executing “hive --service llap …”, I modified ${hive-binary-dist}/scripts/llap/bin/llap-daemon-env.sh to modify LLAP_DAEMON_OPTS.
> - What I found, however, is that ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh refers to ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh, not ${LLAP_DAEMON_BIN_HOME}
> - I also tried to locate llap-daemon-env.sh under ${hive-binary-dost}/conf but found that LlapServiceDriver doesn’t copy that file under LLAP_DAEMON_CONF_DIR.
> - I had to modify ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh to load llap-daemon-env.sh properly by making it refer to ${LLAP_DAEMON_BIN_HOME}/llap-daemon-env.sh instead of ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh.
>
> - eastcircle
>
>
>
>
> --
> ~Rajesh.B
Re: user@tez.apache.org
Posted by Rajesh Balamohan <ra...@gmail.com>.
Much easier option is to make use of
https://github.com/t3rmin4t0r/tez-autobuild (edit/set args in
slider-gen.sh).
~Rajesh.B
On Mon, Dec 26, 2016 at 11:02 AM, Rajesh Balamohan <rb...@apache.org>
wrote:
> Here is an example:
>
> hive --service llap --instances 1 --args "-XX:+UseG1GC
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000"
> --cache 48000m --executors 8 --iothreads 8 --size 180000m --xmx 128000m
> --loglevel INFO --javaHome /usr/jdk64/jdk1.8.0_77/
>
> This would generate a "run.sh" script which can be run to start LLAP. This
> would open up 8000 port that you have mentioned in the machine where LLAP
> daemon is started.
>
> ~Rajesh.B
>
> On Mon, Dec 26, 2016 at 10:37 AM, 김동원 <ea...@gmail.com> wrote:
>
>> I want my IntelliJ to attach to the running LLAP damons by adding the
>> following JVM option (for a debugging purpose):
>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
>>
>> Where is the right place to append the line?
>>
>> My previous attempt was as follows :
>> - Before executing “hive --service llap …”, I modified
>> ${hive-binary-dist}/scripts/llap/bin/llap-daemon-env.sh to modify
>> LLAP_DAEMON_OPTS.
>> - What I found, however, is that ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh
>> refers to ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh, not
>> ${LLAP_DAEMON_BIN_HOME}
>> - I also tried to locate llap-daemon-env.sh under
>> ${hive-binary-dost}/conf but found that LlapServiceDriver doesn’t copy that
>> file under LLAP_DAEMON_CONF_DIR.
>> - I had to modify ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh to
>> load llap-daemon-env.sh properly by making it refer to
>> ${LLAP_DAEMON_BIN_HOME}/llap-daemon-env.sh instead of
>> ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh.
>>
>> - eastcircle
>
>
>
--
~Rajesh.B
Re: user@tez.apache.org
Posted by Rajesh Balamohan <rb...@apache.org>.
Here is an example:
hive --service llap --instances 1 --args "-XX:+UseG1GC
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000" --cache
48000m --executors 8 --iothreads 8 --size 180000m --xmx 128000m --loglevel
INFO --javaHome /usr/jdk64/jdk1.8.0_77/
This would generate a "run.sh" script which can be run to start LLAP. This
would open up 8000 port that you have mentioned in the machine where LLAP
daemon is started.
~Rajesh.B
On Mon, Dec 26, 2016 at 10:37 AM, 김동원 <ea...@gmail.com> wrote:
> I want my IntelliJ to attach to the running LLAP damons by adding the
> following JVM option (for a debugging purpose):
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
>
> Where is the right place to append the line?
>
> My previous attempt was as follows :
> - Before executing “hive --service llap …”, I modified
> ${hive-binary-dist}/scripts/llap/bin/llap-daemon-env.sh to modify
> LLAP_DAEMON_OPTS.
> - What I found, however, is that ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh
> refers to ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh, not
> ${LLAP_DAEMON_BIN_HOME}
> - I also tried to locate llap-daemon-env.sh under ${hive-binary-dost}/conf
> but found that LlapServiceDriver doesn’t copy that file under
> LLAP_DAEMON_CONF_DIR.
> - I had to modify ${hive-binary-dist}/scripts/llap/bin/llapDaemon.sh to
> load llap-daemon-env.sh properly by making it refer to
> ${LLAP_DAEMON_BIN_HOME}/llap-daemon-env.sh instead of
> ${LLAP_DAEMON_CONF_DIR}/llap-daemon-env.sh.
>
> - eastcircle