You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by sam mohel <sa...@gmail.com> on 2016/04/23 01:41:20 UTC
Is ganglia like visualvm ?
I submitted topology and want to measure performance of CPU by like getting
graph
Can I use ganglia ? or visual VM can do it ?
Iam using storm -0.9.6 and it has already visulaization for spout and bolts
. how can I understand it well ? I don't understand numbers that appeared
in the graph . and how can I use it in evaluation of toplogy
Thanks for any help
Re: Is ganglia like visualvm ?
Posted by sam mohel <sa...@gmail.com>.
Thanks for help
On Tuesday, April 26, 2016, Stephen Powis <sp...@salesforce.com> wrote:
> There's no real "right" answer to this, it depends on your hardware,
> utilization of your hardware, and all sorts of other kinds of parameters.
>
> I've never used Ganglia myself, but my naive understanding is you have an
> agent that runs on every node you want to monitor. Then the agents "report
> back" metrics to a central server which provides the view into the metrics
> of all your nodes (ie all the graphs and what not).
>
> So if it was me, I'd install the ganglia agent on every node I want to
> monitor. Where to install the central server? You'll have to read up the
> documentation on Ganglia and figure out where that fits best within the
> servers you own. You'd find the best info about Ganglia on their website
> docs, or their mailing list.
>
> On Mon, Apr 25, 2016 at 8:35 PM, sam mohel <sammohel5@gmail.com
> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>> wrote:
>
>> Can you help please ?
>>
>>
>> On Monday, April 25, 2016, sam mohel <sammohel5@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>> wrote:
>>
>>> Thanks for helping , it helped me a lot
>>> But how can i set up Ganglia i mean i have two machines
>>> Should i install ganglia in machine 1 which i run on it nimbus or on
>>> worker machine ?
>>>
>>>
>>> On Mon, Apr 25, 2016 at 12:29 AM, Manu Zhang <ow...@gmail.com>
>>> wrote:
>>>
>>>> Hi Sam,
>>>>
>>>> Generally, Ganglia gives usage of CPU, Memory, Disk and Network of the
>>>> whole cluster while visual VM shows CPU and Memory metrics per JVM.
>>>>
>>>> If the topology is the only running job on your cluster, Ganglia will
>>>> let you know whether your topology has fully utilized the resources of the
>>>> cluster and where the bottleneck is. For example, is the CPU abnormally
>>>> high for very simple logics ? is the GbE network already full ?
>>>>
>>>> With visual VM, you might be able to pinpoint the problem. It shows the
>>>> CPU condition of each thread at runtime, whether it's running, sleeping or
>>>> blocking and the thread can be dumped. Plus, you can also do sampling or
>>>> profiling for CPU / memory which can help you to find the hot spots in your
>>>> code.
>>>>
>>>> Hope this helps.
>>>>
>>>> Regards,
>>>> Manu
>>>>
>>>> On Sun, Apr 24, 2016 at 3:20 PM sam mohel <sa...@gmail.com> wrote:
>>>>
>>>>> Can anybody help ?
>>>>>
>>>>> On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
>>>>>
>>>>> > I submitted topology and want to measure performance of CPU by like
>>>>> > getting graph
>>>>> >
>>>>> > Can I use ganglia ? or visual VM can do it ?
>>>>> >
>>>>> > Iam using storm -0.9.6 and it has already visulaization for spout and
>>>>> > bolts . how can I understand it well ? I don't understand numbers
>>>>> that
>>>>> > appeared in the graph . and how can I use it in evaluation of toplogy
>>>>> >
>>>>> > Thanks for any help
>>>>> >
>>>>>
>>>>
>>>
>
Re: Is ganglia like visualvm ?
Posted by Stephen Powis <sp...@salesforce.com>.
There's no real "right" answer to this, it depends on your hardware,
utilization of your hardware, and all sorts of other kinds of parameters.
I've never used Ganglia myself, but my naive understanding is you have an
agent that runs on every node you want to monitor. Then the agents "report
back" metrics to a central server which provides the view into the metrics
of all your nodes (ie all the graphs and what not).
So if it was me, I'd install the ganglia agent on every node I want to
monitor. Where to install the central server? You'll have to read up the
documentation on Ganglia and figure out where that fits best within the
servers you own. You'd find the best info about Ganglia on their website
docs, or their mailing list.
On Mon, Apr 25, 2016 at 8:35 PM, sam mohel <sa...@gmail.com> wrote:
> Can you help please ?
>
>
> On Monday, April 25, 2016, sam mohel <sa...@gmail.com> wrote:
>
>> Thanks for helping , it helped me a lot
>> But how can i set up Ganglia i mean i have two machines
>> Should i install ganglia in machine 1 which i run on it nimbus or on
>> worker machine ?
>>
>>
>> On Mon, Apr 25, 2016 at 12:29 AM, Manu Zhang <ow...@gmail.com>
>> wrote:
>>
>>> Hi Sam,
>>>
>>> Generally, Ganglia gives usage of CPU, Memory, Disk and Network of the
>>> whole cluster while visual VM shows CPU and Memory metrics per JVM.
>>>
>>> If the topology is the only running job on your cluster, Ganglia will
>>> let you know whether your topology has fully utilized the resources of the
>>> cluster and where the bottleneck is. For example, is the CPU abnormally
>>> high for very simple logics ? is the GbE network already full ?
>>>
>>> With visual VM, you might be able to pinpoint the problem. It shows the
>>> CPU condition of each thread at runtime, whether it's running, sleeping or
>>> blocking and the thread can be dumped. Plus, you can also do sampling or
>>> profiling for CPU / memory which can help you to find the hot spots in your
>>> code.
>>>
>>> Hope this helps.
>>>
>>> Regards,
>>> Manu
>>>
>>> On Sun, Apr 24, 2016 at 3:20 PM sam mohel <sa...@gmail.com> wrote:
>>>
>>>> Can anybody help ?
>>>>
>>>> On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
>>>>
>>>> > I submitted topology and want to measure performance of CPU by like
>>>> > getting graph
>>>> >
>>>> > Can I use ganglia ? or visual VM can do it ?
>>>> >
>>>> > Iam using storm -0.9.6 and it has already visulaization for spout and
>>>> > bolts . how can I understand it well ? I don't understand numbers that
>>>> > appeared in the graph . and how can I use it in evaluation of toplogy
>>>> >
>>>> > Thanks for any help
>>>> >
>>>>
>>>
>>
Re: Is ganglia like visualvm ?
Posted by sam mohel <sa...@gmail.com>.
Can you help please ?
On Monday, April 25, 2016, sam mohel <sa...@gmail.com> wrote:
> Thanks for helping , it helped me a lot
> But how can i set up Ganglia i mean i have two machines
> Should i install ganglia in machine 1 which i run on it nimbus or on
> worker machine ?
>
>
> On Mon, Apr 25, 2016 at 12:29 AM, Manu Zhang <owenzhang1990@gmail.com
> <javascript:_e(%7B%7D,'cvml','owenzhang1990@gmail.com');>> wrote:
>
>> Hi Sam,
>>
>> Generally, Ganglia gives usage of CPU, Memory, Disk and Network of the
>> whole cluster while visual VM shows CPU and Memory metrics per JVM.
>>
>> If the topology is the only running job on your cluster, Ganglia will let
>> you know whether your topology has fully utilized the resources of the
>> cluster and where the bottleneck is. For example, is the CPU abnormally
>> high for very simple logics ? is the GbE network already full ?
>>
>> With visual VM, you might be able to pinpoint the problem. It shows the
>> CPU condition of each thread at runtime, whether it's running, sleeping or
>> blocking and the thread can be dumped. Plus, you can also do sampling or
>> profiling for CPU / memory which can help you to find the hot spots in your
>> code.
>>
>> Hope this helps.
>>
>> Regards,
>> Manu
>>
>> On Sun, Apr 24, 2016 at 3:20 PM sam mohel <sammohel5@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>> wrote:
>>
>>> Can anybody help ?
>>>
>>> On Saturday, April 23, 2016, sam mohel <sammohel5@gmail.com
>>> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>> wrote:
>>>
>>> > I submitted topology and want to measure performance of CPU by like
>>> > getting graph
>>> >
>>> > Can I use ganglia ? or visual VM can do it ?
>>> >
>>> > Iam using storm -0.9.6 and it has already visulaization for spout and
>>> > bolts . how can I understand it well ? I don't understand numbers that
>>> > appeared in the graph . and how can I use it in evaluation of toplogy
>>> >
>>> > Thanks for any help
>>> >
>>>
>>
>
Re: Is ganglia like visualvm ?
Posted by sam mohel <sa...@gmail.com>.
Thanks for helping , it helped me a lot
But how can i set up Ganglia i mean i have two machines
Should i install ganglia in machine 1 which i run on it nimbus or on
worker machine ?
On Mon, Apr 25, 2016 at 12:29 AM, Manu Zhang <ow...@gmail.com>
wrote:
> Hi Sam,
>
> Generally, Ganglia gives usage of CPU, Memory, Disk and Network of the
> whole cluster while visual VM shows CPU and Memory metrics per JVM.
>
> If the topology is the only running job on your cluster, Ganglia will let
> you know whether your topology has fully utilized the resources of the
> cluster and where the bottleneck is. For example, is the CPU abnormally
> high for very simple logics ? is the GbE network already full ?
>
> With visual VM, you might be able to pinpoint the problem. It shows the
> CPU condition of each thread at runtime, whether it's running, sleeping or
> blocking and the thread can be dumped. Plus, you can also do sampling or
> profiling for CPU / memory which can help you to find the hot spots in your
> code.
>
> Hope this helps.
>
> Regards,
> Manu
>
> On Sun, Apr 24, 2016 at 3:20 PM sam mohel <sa...@gmail.com> wrote:
>
>> Can anybody help ?
>>
>> On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
>>
>> > I submitted topology and want to measure performance of CPU by like
>> > getting graph
>> >
>> > Can I use ganglia ? or visual VM can do it ?
>> >
>> > Iam using storm -0.9.6 and it has already visulaization for spout and
>> > bolts . how can I understand it well ? I don't understand numbers that
>> > appeared in the graph . and how can I use it in evaluation of toplogy
>> >
>> > Thanks for any help
>> >
>>
>
Re: Is ganglia like visualvm ?
Posted by Manu Zhang <ow...@gmail.com>.
Hi Sam,
Generally, Ganglia gives usage of CPU, Memory, Disk and Network of the
whole cluster while visual VM shows CPU and Memory metrics per JVM.
If the topology is the only running job on your cluster, Ganglia will let
you know whether your topology has fully utilized the resources of the
cluster and where the bottleneck is. For example, is the CPU abnormally
high for very simple logics ? is the GbE network already full ?
With visual VM, you might be able to pinpoint the problem. It shows the CPU
condition of each thread at runtime, whether it's running, sleeping or
blocking and the thread can be dumped. Plus, you can also do sampling or
profiling for CPU / memory which can help you to find the hot spots in your
code.
Hope this helps.
Regards,
Manu
On Sun, Apr 24, 2016 at 3:20 PM sam mohel <sa...@gmail.com> wrote:
> Can anybody help ?
>
> On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
>
> > I submitted topology and want to measure performance of CPU by like
> > getting graph
> >
> > Can I use ganglia ? or visual VM can do it ?
> >
> > Iam using storm -0.9.6 and it has already visulaization for spout and
> > bolts . how can I understand it well ? I don't understand numbers that
> > appeared in the graph . and how can I use it in evaluation of toplogy
> >
> > Thanks for any help
> >
>
Re: Is ganglia like visualvm ?
Posted by Manu Zhang <ow...@gmail.com>.
Hi Sam,
Generally, Ganglia gives usage of CPU, Memory, Disk and Network of the
whole cluster while visual VM shows CPU and Memory metrics per JVM.
If the topology is the only running job on your cluster, Ganglia will let
you know whether your topology has fully utilized the resources of the
cluster and where the bottleneck is. For example, is the CPU abnormally
high for very simple logics ? is the GbE network already full ?
With visual VM, you might be able to pinpoint the problem. It shows the CPU
condition of each thread at runtime, whether it's running, sleeping or
blocking and the thread can be dumped. Plus, you can also do sampling or
profiling for CPU / memory which can help you to find the hot spots in your
code.
Hope this helps.
Regards,
Manu
On Sun, Apr 24, 2016 at 3:20 PM sam mohel <sa...@gmail.com> wrote:
> Can anybody help ?
>
> On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
>
> > I submitted topology and want to measure performance of CPU by like
> > getting graph
> >
> > Can I use ganglia ? or visual VM can do it ?
> >
> > Iam using storm -0.9.6 and it has already visulaization for spout and
> > bolts . how can I understand it well ? I don't understand numbers that
> > appeared in the graph . and how can I use it in evaluation of toplogy
> >
> > Thanks for any help
> >
>
Re: Is ganglia like visualvm ?
Posted by sam mohel <sa...@gmail.com>.
Thanks for replying , sorry i'm new for using this tools and in storm ,
what is the relation that this tool gave me ? i mean the graph that i wil
get it between what ? and how can i get benefit from it ?
On Sun, Apr 24, 2016 at 5:36 PM, Stephen Powis <sp...@salesforce.com>
wrote:
> I use the following metric reporting plugin to send metrics to graphite,
> and use grafana to visualize them.
>
> https://github.com/verisign/storm-graphite
>
> On Sun, Apr 24, 2016 at 3:20 AM, sam mohel <sa...@gmail.com> wrote:
>
>> Can anybody help ?
>>
>> On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
>>
>>> I submitted topology and want to measure performance of CPU by like
>>> getting graph
>>>
>>> Can I use ganglia ? or visual VM can do it ?
>>>
>>> Iam using storm -0.9.6 and it has already visulaization for spout and
>>> bolts . how can I understand it well ? I don't understand numbers that
>>> appeared in the graph . and how can I use it in evaluation of toplogy
>>>
>>> Thanks for any help
>>>
>>
>
Re: Is ganglia like visualvm ?
Posted by Stephen Powis <sp...@salesforce.com>.
I use the following metric reporting plugin to send metrics to graphite,
and use grafana to visualize them.
https://github.com/verisign/storm-graphite
On Sun, Apr 24, 2016 at 3:20 AM, sam mohel <sa...@gmail.com> wrote:
> Can anybody help ?
>
> On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
>
>> I submitted topology and want to measure performance of CPU by like
>> getting graph
>>
>> Can I use ganglia ? or visual VM can do it ?
>>
>> Iam using storm -0.9.6 and it has already visulaization for spout and
>> bolts . how can I understand it well ? I don't understand numbers that
>> appeared in the graph . and how can I use it in evaluation of toplogy
>>
>> Thanks for any help
>>
>
Re: Is ganglia like visualvm ?
Posted by sam mohel <sa...@gmail.com>.
Can anybody help ?
On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
> I submitted topology and want to measure performance of CPU by like
> getting graph
>
> Can I use ganglia ? or visual VM can do it ?
>
> Iam using storm -0.9.6 and it has already visulaization for spout and
> bolts . how can I understand it well ? I don't understand numbers that
> appeared in the graph . and how can I use it in evaluation of toplogy
>
> Thanks for any help
>
Re: Is ganglia like visualvm ?
Posted by sam mohel <sa...@gmail.com>.
Can anybody help ?
On Saturday, April 23, 2016, sam mohel <sa...@gmail.com> wrote:
> I submitted topology and want to measure performance of CPU by like
> getting graph
>
> Can I use ganglia ? or visual VM can do it ?
>
> Iam using storm -0.9.6 and it has already visulaization for spout and
> bolts . how can I understand it well ? I don't understand numbers that
> appeared in the graph . and how can I use it in evaluation of toplogy
>
> Thanks for any help
>