You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by sam mohel <sa...@gmail.com> on 2016/05/10 21:20:58 UTC
Storm cluster with time
Iam a researcher and I hope you can help me on this questions my goal is
to update algorithm that coder used to make data more accuracy. I have one
Java process that make me need one supervisor right ? When I made machine
core i5 with 8 ram as supervisor and machine core2duo with 2 ram as nimbus
I got error in GC overhead as data is huge and need more ram . I swapped
the two machines then everything worked but took long time . It took more
than 20 hours . Is it normal ? Or what can I do ?
Re: Storm cluster with time
Posted by Sai Dilip Reddy Kiralam <dk...@aadhya-analytics.com>.
HI sam,
when you have a bottleneck issue your spout sats in UI shows the failed
tuples or your bolts will turn into red colour in visualisation view of
your topology.
when you note such changes in UI then your topology is having bottleneck.
To avoid this bottle-necks you just need to increase parallelism of bolts.
*Best regards,*
*K.Sai Dilip Reddy.*
On Wed, May 11, 2016 at 3:52 AM, sam mohel <sa...@gmail.com> wrote:
> Thanks for replying . But how can I find bottleneck from storm ui . And
> what can i do if I find it ?
>
> On Wednesday, May 11, 2016, cogumelosmaravilha <co...@sapo.pt>
> wrote:
>
>> You can check where is the bottleneck using the ui of Storm.
>>
>> On 10-05-2016 22:20, sam mohel wrote:
>>
>> Iam a researcher and I hope you can help me on this questions my goal
>> is to update algorithm that coder used to make data more accuracy. I have
>> one Java process that make me need one supervisor right ? When I made
>> machine core i5 with 8 ram as supervisor and machine core2duo with 2 ram as
>> nimbus I got error in GC overhead as data is huge and need more ram . I
>> swapped the two machines then everything worked but took long time . It
>> took more than 20 hours . Is it normal ? Or what can I do ?
>>
>>
>>
Re: Storm cluster with time
Posted by cogumelosmaravilha <co...@sapo.pt>.
In the example topology there is no bottleneck. Try find it in yours!
Google storm latency.
On 11-05-2016 22:28, sam mohel wrote:
> Thanks gor helping . But excuse me I didn't get what big values you
> mean in visualization graph ?
>
> On Wednesday, May 11, 2016, <cogumelosmaravilha@sapo.pt
> <ma...@sapo.pt>> wrote:
>
> Check the image. Where you find big values there's the bottleneck.
>
> Citando sam mohel <sammohel5@gmail.com
> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>>:
>
>> Thanks for replying . But how can I find bottleneck from storm ui
>> . And what can i do if I find it ?
>>
>>
>> On Wednesday, May 11, 2016, cogumelosmaravilha
>> <cogumelosmaravilha@sapo.pt
>> <javascript:_e(%7B%7D,'cvml','cogumelosmaravilha@sapo.pt');>> wrote:
>>
>> You can check where is the bottleneck using the ui of Storm.
>> On 10-05-2016 22:20, sam mohel wrote:
>>>
>>> Iam a researcher and I hope you can help me on this
>>> questions my goal is to update algorithm that coder used to
>>> make data more accuracy. I have one Java process that make
>>> me need one supervisor right ? When I made machine core i5
>>> with 8 ram as supervisor and machine core2duo with 2 ram as
>>> nimbus I got error in GC overhead as data is huge and need
>>> more ram . I swapped the two machines then everything worked
>>> but took long time . It took more than 20 hours . Is it
>>> normal ? Or what can I do ?
>>>
>
Re: Storm cluster with time
Posted by sam mohel <sa...@gmail.com>.
Thanks gor helping . But excuse me I didn't get what big values you mean in
visualization graph ?
On Wednesday, May 11, 2016, <co...@sapo.pt> wrote:
> Check the image. Where you find big values there's the bottleneck.
>
>
> Citando sam mohel <sammohel5@gmail.com
> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>>:
>
> Thanks for replying . But how can I find bottleneck from storm ui . And
> what can i do if I find it ?
>
> On Wednesday, May 11, 2016, cogumelosmaravilha <cogumelosmaravilha@sapo.pt
> <javascript:_e(%7B%7D,'cvml','cogumelosmaravilha@sapo.pt');>> wrote:
>>
>> You can check where is the bottleneck using the ui of Storm.
>>
>> On 10-05-2016 22:20, sam mohel wrote:
>>
>> Iam a researcher and I hope you can help me on this questions my goal
>> is to update algorithm that coder used to make data more accuracy. I have
>> one Java process that make me need one supervisor right ? When I made
>> machine core i5 with 8 ram as supervisor and machine core2duo with 2 ram as
>> nimbus I got error in GC overhead as data is huge and need more ram . I
>> swapped the two machines then everything worked but took long time . It
>> took more than 20 hours . Is it normal ? Or what can I do ?
>>
>>
>
>
Re: Storm cluster with time
Posted by sam mohel <sa...@gmail.com>.
Thanks for replying . Values I got capacity column are 0.019 and 0.3 is the
largest I got . Do you mean by near to "1" like 0.9
What if I have in bolt 0.000 ?
On Wednesday, May 11, 2016, Sai Dilip Reddy Kiralam <
dkiralam@aadhya-analytics.com> wrote:
> check last 10m capacity in bolts sats & if the value is near to 1 then
> that bolt is black sheep to your topology increase the parallelism to that
> bolt.
>
>
>
> *Best regards,*
>
> *K.Sai Dilip Reddy.*
>
> On Wed, May 11, 2016 at 1:43 PM, <cogumelosmaravilha@sapo.pt
> <javascript:_e(%7B%7D,'cvml','cogumelosmaravilha@sapo.pt');>> wrote:
>
>> Check the image. Where you find big values there's the bottleneck.
>>
>>
>> Citando sam mohel <sammohel5@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>>:
>>
>> Thanks for replying . But how can I find bottleneck from storm ui . And
>> what can i do if I find it ?
>>
>> On Wednesday, May 11, 2016, cogumelosmaravilha <
>> cogumelosmaravilha@sapo.pt
>> <javascript:_e(%7B%7D,'cvml','cogumelosmaravilha@sapo.pt');>> wrote:
>>>
>>> You can check where is the bottleneck using the ui of Storm.
>>>
>>> On 10-05-2016 22:20, sam mohel wrote:
>>>
>>> Iam a researcher and I hope you can help me on this questions my goal
>>> is to update algorithm that coder used to make data more accuracy. I have
>>> one Java process that make me need one supervisor right ? When I made
>>> machine core i5 with 8 ram as supervisor and machine core2duo with 2 ram as
>>> nimbus I got error in GC overhead as data is huge and need more ram . I
>>> swapped the two machines then everything worked but took long time . It
>>> took more than 20 hours . Is it normal ? Or what can I do ?
>>>
>>>
>>
>>
>
>
Re: Storm cluster with time
Posted by Sai Dilip Reddy Kiralam <dk...@aadhya-analytics.com>.
check last 10m capacity in bolts sats & if the value is near to 1 then that
bolt is black sheep to your topology increase the parallelism to that
bolt.
*Best regards,*
*K.Sai Dilip Reddy.*
On Wed, May 11, 2016 at 1:43 PM, <co...@sapo.pt> wrote:
> Check the image. Where you find big values there's the bottleneck.
>
>
> Citando sam mohel <sa...@gmail.com>:
>
> Thanks for replying . But how can I find bottleneck from storm ui . And
> what can i do if I find it ?
>
> On Wednesday, May 11, 2016, cogumelosmaravilha <co...@sapo.pt>
> wrote:
>>
>> You can check where is the bottleneck using the ui of Storm.
>>
>> On 10-05-2016 22:20, sam mohel wrote:
>>
>> Iam a researcher and I hope you can help me on this questions my goal
>> is to update algorithm that coder used to make data more accuracy. I have
>> one Java process that make me need one supervisor right ? When I made
>> machine core i5 with 8 ram as supervisor and machine core2duo with 2 ram as
>> nimbus I got error in GC overhead as data is huge and need more ram . I
>> swapped the two machines then everything worked but took long time . It
>> took more than 20 hours . Is it normal ? Or what can I do ?
>>
>>
>
>
Re: Storm cluster with time
Posted by co...@sapo.pt.
Check the image. Where you find big values there's the bottleneck.
�
Citando sam mohel <sa...@gmail.com>:
> Thanks for replying . But how can I find bottleneck from storm ui .
> And what can i do if I find it ?�
>
> On Wednesday, May 11, 2016, cogumelosmaravilha
> <co...@sapo.pt> wrote:
>> You can check where is the bottleneck using the ui of Storm.
>> � On 10-05-2016 22:20, sam mohel wrote:
>>
>>> �Iam a researcher and I hope �you can help me on this questions my
>>> goal is to update algorithm that coder used to make data more
>>> accuracy. I have one Java process that make me need one supervisor
>>> right ? When I made machine core i5 with 8 ram as supervisor and
>>> machine core2duo with 2 ram as nimbus I got error in GC overhead
>>> as data is huge and need more ram . I swapped the two machines
>>> then everything worked but took long time . It took more than 20
>>> hours . Is it normal ? Or what can I do ?
�
Re: Storm cluster with time
Posted by sam mohel <sa...@gmail.com>.
Thanks for replying . But how can I find bottleneck from storm ui . And
what can i do if I find it ?
On Wednesday, May 11, 2016, cogumelosmaravilha <co...@sapo.pt>
wrote:
> You can check where is the bottleneck using the ui of Storm.
>
> On 10-05-2016 22:20, sam mohel wrote:
>
> Iam a researcher and I hope you can help me on this questions my goal is
> to update algorithm that coder used to make data more accuracy. I have one
> Java process that make me need one supervisor right ? When I made machine
> core i5 with 8 ram as supervisor and machine core2duo with 2 ram as nimbus
> I got error in GC overhead as data is huge and need more ram . I swapped
> the two machines then everything worked but took long time . It took more
> than 20 hours . Is it normal ? Or what can I do ?
>
>
>
Re: Storm cluster with time
Posted by cogumelosmaravilha <co...@sapo.pt>.
You can check where is the bottleneck using the ui of Storm.
On 10-05-2016 22:20, sam mohel wrote:
> Iam a researcher and I hope you can help me on this questions my
> goal is to update algorithm that coder used to make data more
> accuracy. I have one Java process that make me need one supervisor
> right ? When I made machine core i5 with 8 ram as supervisor and
> machine core2duo with 2 ram as nimbus I got error in GC overhead as
> data is huge and need more ram . I swapped the two machines then
> everything worked but took long time . It took more than 20 hours . Is
> it normal ? Or what can I do ?