Can i increase the parallelism of the topology as per traffic

["M.Tarkeshwar Rao" <ta...@gmail.com>]

Hi,

Can i increase the parallelism of the topology as per traffic?

I need inputs from you all. any link?

(Hint: find the bolt stats through nimbus service and increase the bolt’s
executor by firing rebalancing command after fixed interval if load
increases)

regards
tarkesh

Re: Can i increase the parallelism of the topology as per traffic

[Nathan Leung <nc...@gmail.com>]

Not to be trite, but the help message for the command will tell you
everything you need:

$ storm help rebalance
Syntax: [storm rebalance topology-name [-w wait-time-secs] [-n
new-num-workers] [-e component=parallelism]*]

    Sometimes you may wish to spread out where the workers for a topology
    are running. For example, let's say you have a 10 node cluster running
    4 workers per node, and then let's say you add another 10 nodes to
    the cluster. You may wish to have Storm spread out the workers for the
    running topology so that each node runs 2 workers. One way to do this
    is to kill the topology and resubmit it, but Storm provides a
"rebalance"
    command that provides an easier way to do this.

    Rebalance will first deactivate the topology for the duration of the
    message timeout (overridable with the -w flag) and then redistribute
    the workers evenly around the cluster. The topology will then return to
    its previous state of activation (so a deactivated topology will still
    be deactivated and an activated topology will go back to being
activated).

    The rebalance command can also be used to change the parallelism of a
running topology.
    Use the -n and -e switches to change the number of workers or number of
executors of a component
    respectively.




On Wed, May 7, 2014 at 1:44 PM, Sajith <sa...@gmail.com> wrote:

> Hi Nathan,
>
> Can you please elaborate on how to do that?
>
> Thanks,
> Sajith.
>
>
> On Tue, May 6, 2014 at 5:01 PM, Nathan Leung <nc...@gmail.com> wrote:
>
>> The number of executors and workers can be changed but the number of
>> tasks is fixed at topology creation time.
>>  On May 6, 2014 1:40 AM, "M.Tarkeshwar Rao" <ta...@gmail.com>
>> wrote:
>>
>>>
>>> Hi,
>>>
>>> Can i increase the parallelism of the topology as per traffic?
>>>
>>> I need inputs from you all. any link?
>>>
>>> (Hint: find the bolt stats through nimbus service and increase the
>>> bolt’s executor by firing rebalancing command after fixed interval if
>>> load increases)
>>>
>>> regards
>>> tarkesh
>>>
>>>
>

Re: Can i increase the parallelism of the topology as per traffic

[Sajith <sa...@gmail.com>]

Hi Nathan,

Can you please elaborate on how to do that?

Thanks,
Sajith.


On Tue, May 6, 2014 at 5:01 PM, Nathan Leung <nc...@gmail.com> wrote:

> The number of executors and workers can be changed but the number of tasks
> is fixed at topology creation time.
> On May 6, 2014 1:40 AM, "M.Tarkeshwar Rao" <ta...@gmail.com> wrote:
>
>>
>> Hi,
>>
>> Can i increase the parallelism of the topology as per traffic?
>>
>> I need inputs from you all. any link?
>>
>> (Hint: find the bolt stats through nimbus service and increase the bolt’s
>> executor by firing rebalancing command after fixed interval if load
>> increases)
>>
>> regards
>> tarkesh
>>
>>

Re: Can i increase the parallelism of the topology as per traffic

[Nathan Leung <nc...@gmail.com>]

The number of executors and workers can be changed but the number of tasks
is fixed at topology creation time.
On May 6, 2014 1:40 AM, "M.Tarkeshwar Rao" <ta...@gmail.com> wrote:

>
> Hi,
>
> Can i increase the parallelism of the topology as per traffic?
>
> I need inputs from you all. any link?
>
> (Hint: find the bolt stats through nimbus service and increase the bolt’s
> executor by firing rebalancing command after fixed interval if load
> increases)
>
> regards
> tarkesh
>
>