You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@storm.apache.org by Julien Nioche <li...@gmail.com> on 2018/03/19 11:29:49 UTC

Worker profiling on 1.2.1

Hi,

I am using Storm 1.2.1 and want to profile my workers. I've set
*worker.profiler.enabled* to true in the config and the UI's showing the
values below so everything should be fine. I've modified the childopts so
that it works with Oracle Java 9 and flight.bash works when I run it
manually.

worker.profiler.childopts

"-XX:+UnlockCommercialFeatures -XX:+FlightRecorder"

worker.profiler.command

"flight.bash"

worker.profiler.enabled

true


I have 'Jstack', 'Restart worker' and 'Heap' available as actions and
clickable when I select a worker but there is nothing about profiling. The
documentation on
http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html
<http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html> does
not mention anything specific.

Am I missing something here?

Thanks

Julien

-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>

Re: Worker profiling on 1.2.1

Posted by Roshan Naik <ro...@yahoo.com>.

What I suggested is an alternative if you are running into issues with. It will enable profiling on startup. With this method you will have to manually fire up the profiler to attach it to the worker.. and not via the web UI buttons.

Sent from Yahoo Mail for iPhone

On Thursday, March 22, 2018, 2:05 AM, Julien Nioche <li...@gmail.com> wrote:

Hi Roshan,
Whether I change it in the topology config or in the main one should not make any difference, should it? And that doesn't explain why I am not seeing the start profiling option at all?
Thanks
Julien
On 22 March 2018 at 06:26, Roshan Naik <ro...@yahoo.com> wrote:

you can set any java options to enable java profilers via the topology.worker.childopts when submitting the topology.

-Roshan

Sent from Yahoo Mail for iPhone

On Monday, March 19, 2018, 4:29 AM, Julien Nioche <lists.digitalpebble@gmail.com > wrote:

Hi,
I am using Storm 1.2.1 and want to profile my workers. I've set worker.profiler.enabled to true in the config and the UI's showing the values below so everything should be fine. I've modified the childopts so that it works with Oracle Java 9 and flight.bash works when I run it manually.

I have 'Jstack', 'Restart worker' and 'Heap' available as actions and clickable when I select a worker but there is nothing about profiling. The documentation on http://storm.apache.org/ releases/1.2.1/dynamic-worker- profiling.html does not mention anything specific.
Am I missing something here?
Thanks
Julien

Open Source Solutions for Text Engineering
http://www.digitalpebble.com
http://digitalpebble.blogspot. com/
#digitalpebble

Open Source Solutions for Text Engineering
http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble

Re: Worker profiling on 1.2.1

Posted by Julien Nioche <li...@gmail.com>.

Hi Roshan,

Whether I change it in the topology config or in the main one should not
make any difference, should it? And that doesn't explain why I am not
seeing the start profiling option at all?

Thanks

Julien



On 22 March 2018 at 06:26, Roshan Naik <ro...@yahoo.com> wrote:

>  you can set any java options to enable java profilers via the
> topology.worker.childopts when submitting the topology.
>
> -Roshan
>
> Sent from Yahoo Mail for iPhone
> <https://overview.mail.yahoo.com/?.src=iOS>
>
>
> On Monday, March 19, 2018, 4:29 AM, Julien Nioche <
> lists.digitalpebble@gmail.com> wrote:
>
> Hi,
>
> I am using Storm 1.2.1 and want to profile my workers. I've set
> *worker.profiler.enabled* to true in the config and the UI's showing the
> values below so everything should be fine. I've modified the childopts so
> that it works with Oracle Java 9 and flight.bash works when I run it
> manually.
>
> worker.profiler.childopts
>
> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder"
>
> worker.profiler.command
>
> "flight.bash"
>
> worker.profiler.enabled
>
> true
>
>
> I have 'Jstack', 'Restart worker' and 'Heap' available as actions and
> clickable when I select a worker but there is nothing about profiling. The
> documentation on http://storm.apache.org/releases/1.2.1/dynamic-worker-
> profiling.html
> <http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html> does
> not mention anything specific.
>
> Am I missing something here?
>
> Thanks
>
> Julien
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
>
>


-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>

Re: Worker profiling on 1.2.1

Posted by Roshan Naik <ro...@yahoo.com>.

 you can set any java options to enable java profilers via the topology.worker.childopts when submitting the topology. 

-Roshan 

Sent from Yahoo Mail for iPhone


On Monday, March 19, 2018, 4:29 AM, Julien Nioche <li...@gmail.com> wrote:

Hi,
I am using Storm 1.2.1 and want to profile my workers. I've set worker.profiler.enabled to true in the config and the UI's showing the values below so everything should be fine. I've modified the childopts so that it works with Oracle Java 9 and flight.bash works when I run it manually.

| worker.profiler.childopts | "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder" |
| worker.profiler.command | "flight.bash" |
| worker.profiler.enabled | true |


I have 'Jstack', 'Restart worker' and 'Heap' available as actions and clickable when I select a worker but there is nothing about profiling. The documentation on http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html  does not mention anything specific.
Am I missing something here?
Thanks
Julien

-- 

Open Source Solutions for Text Engineering
http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble

Re: Worker profiling on 1.2.1

Posted by Jungtaek Lim <ka...@gmail.com>.

Thanks Julien for providing the patch! Will take a look.

2018년 3월 27일 (화) 오후 10:46, Julien Nioche <li...@gmail.com>님이
작성:

> Done, see https://github.com/apache/storm/pull/2607
>
> On 27 March 2018 at 11:17, Julien Nioche <li...@gmail.com>
> wrote:
>
>> Hi Jungtaek
>>
>> I had to move to a different server and can't reproduce the same issue
>> anymore. I can now see the 'start' option on the UI.
>>
>> I had to modify the *flight.bash* script as $JAVA_HOME is undefined,
>> this yields
>>
>> */usr/share/apache-storm-1.2.1/bin/flight.bash: line 62: /bin/jstack: No
>> such file or directory*
>>
>> however, the  files can be found in the default path */usr/bin*
>>
>> I will submit a PR so that the script checks that $JDKPATH is not empty
>> and exist before using it for BINPATH. I can't see any downsides in doing
>> so.
>>
>> Thanks
>>
>> Julien
>>
>>
>> On 27 March 2018 at 09:24, Jungtaek Lim <ka...@gmail.com> wrote:
>>
>>> Julien,
>>>
>>> could you check whether 'profilingAndDebuggingCapable' and 'profileActionEnabled'
>>> are true at the result of component API?
>>>
>>> You can check it via opening developer tool in browser, and open
>>> component page, open network tab, and find the request which path is like
>>> '/api/v1/topology/:id/component/:component'.
>>>
>>> http://storm.apache.org/releases/1.2.1/STORM-UI-REST-API.html
>>>
>>> And which OS does Storm is running on?
>>>
>>> Thanks,
>>> Jungtaek Lim (HeartSaVioR)
>>>
>>> 2018년 3월 19일 (월) 오후 8:29, Julien Nioche <li...@gmail.com>님이
>>> 작성:
>>>
>>>> Hi,
>>>>
>>>> I am using Storm 1.2.1 and want to profile my workers. I've set
>>>> *worker.profiler.enabled* to true in the config and the UI's showing
>>>> the values below so everything should be fine. I've modified the
>>>> childopts so that it works with Oracle Java 9 and flight.bash works when I
>>>> run it manually.
>>>>
>>>> worker.profiler.childopts
>>>>
>>>> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder"
>>>>
>>>> worker.profiler.command
>>>>
>>>> "flight.bash"
>>>>
>>>> worker.profiler.enabled
>>>>
>>>> true
>>>>
>>>>
>>>> I have 'Jstack', 'Restart worker' and 'Heap' available as actions and
>>>> clickable when I select a worker but there is nothing about profiling. The
>>>> documentation on
>>>> http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html
>>>> <http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html> does
>>>> not mention anything specific.
>>>>
>>>> Am I missing something here?
>>>>
>>>> Thanks
>>>>
>>>> Julien
>>>>
>>>> --
>>>>
>>>> *Open Source Solutions for Text Engineering*
>>>>
>>>> http://www.digitalpebble.com
>>>> http://digitalpebble.blogspot.com/
>>>> #digitalpebble <http://twitter.com/digitalpebble>
>>>>
>>>
>>
>>
>> --
>>
>> *Open Source Solutions for Text Engineering*
>>
>> http://www.digitalpebble.com
>> http://digitalpebble.blogspot.com/
>> #digitalpebble <http://twitter.com/digitalpebble>
>>
>
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
>

Re: Worker profiling on 1.2.1

Posted by Julien Nioche <li...@gmail.com>.

Done, see https://github.com/apache/storm/pull/2607

On 27 March 2018 at 11:17, Julien Nioche <li...@gmail.com>
wrote:

> Hi Jungtaek
>
> I had to move to a different server and can't reproduce the same issue
> anymore. I can now see the 'start' option on the UI.
>
> I had to modify the *flight.bash* script as $JAVA_HOME is undefined, this
> yields
>
> */usr/share/apache-storm-1.2.1/bin/flight.bash: line 62: /bin/jstack: No
> such file or directory*
>
> however, the  files can be found in the default path */usr/bin*
>
> I will submit a PR so that the script checks that $JDKPATH is not empty
> and exist before using it for BINPATH. I can't see any downsides in doing
> so.
>
> Thanks
>
> Julien
>
>
> On 27 March 2018 at 09:24, Jungtaek Lim <ka...@gmail.com> wrote:
>
>> Julien,
>>
>> could you check whether 'profilingAndDebuggingCapable' and 'profileActionEnabled'
>> are true at the result of component API?
>>
>> You can check it via opening developer tool in browser, and open
>> component page, open network tab, and find the request which path is like
>> '/api/v1/topology/:id/component/:component'.
>>
>> http://storm.apache.org/releases/1.2.1/STORM-UI-REST-API.html
>>
>> And which OS does Storm is running on?
>>
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2018년 3월 19일 (월) 오후 8:29, Julien Nioche <li...@gmail.com>님이
>> 작성:
>>
>>> Hi,
>>>
>>> I am using Storm 1.2.1 and want to profile my workers. I've set
>>> *worker.profiler.enabled* to true in the config and the UI's showing
>>> the values below so everything should be fine. I've modified the
>>> childopts so that it works with Oracle Java 9 and flight.bash works when I
>>> run it manually.
>>>
>>> worker.profiler.childopts
>>>
>>> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder"
>>>
>>> worker.profiler.command
>>>
>>> "flight.bash"
>>>
>>> worker.profiler.enabled
>>>
>>> true
>>>
>>>
>>> I have 'Jstack', 'Restart worker' and 'Heap' available as actions and
>>> clickable when I select a worker but there is nothing about profiling. The
>>> documentation on http://storm.apache.org/rel
>>> eases/1.2.1/dynamic-worker-profiling.html
>>> <http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html> does
>>> not mention anything specific.
>>>
>>> Am I missing something here?
>>>
>>> Thanks
>>>
>>> Julien
>>>
>>> --
>>>
>>> *Open Source Solutions for Text Engineering*
>>>
>>> http://www.digitalpebble.com
>>> http://digitalpebble.blogspot.com/
>>> #digitalpebble <http://twitter.com/digitalpebble>
>>>
>>
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
>



-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>

Re: Worker profiling on 1.2.1

Posted by Julien Nioche <li...@gmail.com>.

Hi Jungtaek

I had to move to a different server and can't reproduce the same issue
anymore. I can now see the 'start' option on the UI.

I had to modify the *flight.bash* script as $JAVA_HOME is undefined, this
yields

*/usr/share/apache-storm-1.2.1/bin/flight.bash: line 62: /bin/jstack: No
such file or directory*

however, the  files can be found in the default path */usr/bin*

I will submit a PR so that the script checks that $JDKPATH is not empty and
exist before using it for BINPATH. I can't see any downsides in doing so.

Thanks

Julien


On 27 March 2018 at 09:24, Jungtaek Lim <ka...@gmail.com> wrote:

> Julien,
>
> could you check whether 'profilingAndDebuggingCapable' and 'profileActionEnabled'
> are true at the result of component API?
>
> You can check it via opening developer tool in browser, and open component
> page, open network tab, and find the request which path is like
> '/api/v1/topology/:id/component/:component'.
>
> http://storm.apache.org/releases/1.2.1/STORM-UI-REST-API.html
>
> And which OS does Storm is running on?
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2018년 3월 19일 (월) 오후 8:29, Julien Nioche <li...@gmail.com>님이
> 작성:
>
>> Hi,
>>
>> I am using Storm 1.2.1 and want to profile my workers. I've set
>> *worker.profiler.enabled* to true in the config and the UI's showing the
>> values below so everything should be fine. I've modified the childopts so
>> that it works with Oracle Java 9 and flight.bash works when I run it
>> manually.
>>
>> worker.profiler.childopts
>>
>> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder"
>>
>> worker.profiler.command
>>
>> "flight.bash"
>>
>> worker.profiler.enabled
>>
>> true
>>
>>
>> I have 'Jstack', 'Restart worker' and 'Heap' available as actions and
>> clickable when I select a worker but there is nothing about profiling. The
>> documentation on http://storm.apache.org/releases/1.2.1/dynamic-worker-
>> profiling.html
>> <http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html> does
>> not mention anything specific.
>>
>> Am I missing something here?
>>
>> Thanks
>>
>> Julien
>>
>> --
>>
>> *Open Source Solutions for Text Engineering*
>>
>> http://www.digitalpebble.com
>> http://digitalpebble.blogspot.com/
>> #digitalpebble <http://twitter.com/digitalpebble>
>>
>


-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>

Re: Worker profiling on 1.2.1

Posted by Jungtaek Lim <ka...@gmail.com>.

Julien,

could you check whether 'profilingAndDebuggingCapable' and
'profileActionEnabled'
are true at the result of component API?

You can check it via opening developer tool in browser, and open component
page, open network tab, and find the request which path is like
'/api/v1/topology/:id/component/:component'.

http://storm.apache.org/releases/1.2.1/STORM-UI-REST-API.html

And which OS does Storm is running on?

Thanks,
Jungtaek Lim (HeartSaVioR)

2018년 3월 19일 (월) 오후 8:29, Julien Nioche <li...@gmail.com>님이
작성:

> Hi,
>
> I am using Storm 1.2.1 and want to profile my workers. I've set
> *worker.profiler.enabled* to true in the config and the UI's showing the
> values below so everything should be fine. I've modified the childopts so
> that it works with Oracle Java 9 and flight.bash works when I run it
> manually.
>
> worker.profiler.childopts
>
> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder"
>
> worker.profiler.command
>
> "flight.bash"
>
> worker.profiler.enabled
>
> true
>
>
> I have 'Jstack', 'Restart worker' and 'Heap' available as actions and
> clickable when I select a worker but there is nothing about profiling. The
> documentation on
> http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html
> <http://storm.apache.org/releases/1.2.1/dynamic-worker-profiling.html> does
> not mention anything specific.
>
> Am I missing something here?
>
> Thanks
>
> Julien
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
>