You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Hemanth Yamijala <yh...@gmail.com> on 2015/06/22 13:40:32 UTC
Rebalancing executors
Hi,
I was testing the rebalance functionality on Storm 0.9.4.
storm rebalance <name> -w 10 -n 2
- Works as expected. It increased the number of workers to 2.
storm rebalance <name> -w 10 -n 2 -e <bolt-name>=20
- Works only for increasing the number of workers, but did *not* change the
number of executors of <bolt-name>.
I came across this question on StackOverflow:
http://stackoverflow.com/questions/18716780/storm-v0-8-2-rebalance-command-not-updating-the-number-of-executors-for-a-bolt
which seems to indicate that the number of executors is bounded by the
number of tasks, and if number of executors = number of tasks (which it is
in my case), we can't increase this further.
Is this right ? Is there a way we can increase the number of executors at
run time without restarting the topology (along with increasing number of
workers) ?
Thanks
Hemanth
Re: Rebalancing executors
Posted by Hemanth Yamijala <yh...@gmail.com>.
Thanks for your response, Matthias. Will try this out and see how it works
for us.
Hemanth
On Mon, Jun 22, 2015 at 6:49 PM, Matthias J. Sax <
mjsax@informatik.hu-berlin.de> wrote:
> >> So, it appears the expectation is to overprovision the number of tasks,
> >> start with minimal number of executors, and then grow executors to
> >> achieve parallelism as workload increases. Is this right ?
>
> Yes. This is correct.
>
> However, over provisioning the number of tasks does not result in
> overhead. If you know the number if distinct key, you can use this
> number (the dop is limited to the number of distinct keys anyway). So
> just set this number high and you are fine (ie, it should not be
> difficult in practice).
>
> -Matthias
>
>
> On 06/22/2015 03:05 PM, Hemanth Yamijala wrote:
> > Hi,
> >
> > Maybe I asked too soon.
> >
> >
> http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/
> >
> > clearly says that the number of tasks is fixed for lifetime of the
> > topology. Number of executors == number of tasks by default. Hence, if
> > the number of executors is already at maximum level, it cannot go
> > further. In the comments section, it further says: "Configuring
> > multiple tasks per executor (thread) gives you the flexibility to expand
> > the topology through the "storm rebalance" command in the future without
> > taking the topology offline. In other words it is not a performance
> reason."
> >
> > So, it appears the expectation is to overprovision the number of tasks,
> > start with minimal number of executors, and then grow executors to
> > achieve parallelism as workload increases. Is this right ?
> >
> > If yes, overall, this is somewhat operationally difficult than expanding
> > on need. Is this a valid use case for Storm to support ? Any plans for
> > this in future ?
> >
> > Thanks
> > hemanth
> >
> >
> >
> > On Mon, Jun 22, 2015 at 5:34 PM, John Yost <soozandjohnyost@gmail.com
> > <ma...@gmail.com>> wrote:
> >
> > This is an excellent question, need this info for myself as well.
> >
> > --John
> >
> > On Mon, Jun 22, 2015 at 7:40 AM, Hemanth Yamijala
> > <yhemanth@gmail.com <ma...@gmail.com>> wrote:
> >
> > Hi,
> >
> > I was testing the rebalance functionality on Storm 0.9.4.
> >
> > storm rebalance <name> -w 10 -n 2
> > - Works as expected. It increased the number of workers to 2.
> >
> > storm rebalance <name> -w 10 -n 2 -e <bolt-name>=20
> > - Works only for increasing the number of workers, but did *not*
> > change the number of executors of <bolt-name>.
> >
> > I came across this question on
> > StackOverflow:
> http://stackoverflow.com/questions/18716780/storm-v0-8-2-rebalance-command-not-updating-the-number-of-executors-for-a-bolt
> >
> > which seems to indicate that the number of executors is bounded
> > by the number of tasks, and if number of executors = number of
> > tasks (which it is in my case), we can't increase this further.
> >
> > Is this right ? Is there a way we can increase the number of
> > executors at run time without restarting the topology (along
> > with increasing number of workers) ?
> >
> > Thanks
> > Hemanth
> >
> >
> >
>
>
Re: Rebalancing executors
Posted by "Matthias J. Sax" <mj...@informatik.hu-berlin.de>.
>> So, it appears the expectation is to overprovision the number of tasks,
>> start with minimal number of executors, and then grow executors to
>> achieve parallelism as workload increases. Is this right ?
Yes. This is correct.
However, over provisioning the number of tasks does not result in
overhead. If you know the number if distinct key, you can use this
number (the dop is limited to the number of distinct keys anyway). So
just set this number high and you are fine (ie, it should not be
difficult in practice).
-Matthias
On 06/22/2015 03:05 PM, Hemanth Yamijala wrote:
> Hi,
>
> Maybe I asked too soon.
>
> http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/
>
> clearly says that the number of tasks is fixed for lifetime of the
> topology. Number of executors == number of tasks by default. Hence, if
> the number of executors is already at maximum level, it cannot go
> further. In the comments section, it further says: "Configuring
> multiple tasks per executor (thread) gives you the flexibility to expand
> the topology through the "storm rebalance" command in the future without
> taking the topology offline. In other words it is not a performance reason."
>
> So, it appears the expectation is to overprovision the number of tasks,
> start with minimal number of executors, and then grow executors to
> achieve parallelism as workload increases. Is this right ?
>
> If yes, overall, this is somewhat operationally difficult than expanding
> on need. Is this a valid use case for Storm to support ? Any plans for
> this in future ?
>
> Thanks
> hemanth
>
>
>
> On Mon, Jun 22, 2015 at 5:34 PM, John Yost <soozandjohnyost@gmail.com
> <ma...@gmail.com>> wrote:
>
> This is an excellent question, need this info for myself as well.
>
> --John
>
> On Mon, Jun 22, 2015 at 7:40 AM, Hemanth Yamijala
> <yhemanth@gmail.com <ma...@gmail.com>> wrote:
>
> Hi,
>
> I was testing the rebalance functionality on Storm 0.9.4.
>
> storm rebalance <name> -w 10 -n 2
> - Works as expected. It increased the number of workers to 2.
>
> storm rebalance <name> -w 10 -n 2 -e <bolt-name>=20
> - Works only for increasing the number of workers, but did *not*
> change the number of executors of <bolt-name>.
>
> I came across this question on
> StackOverflow: http://stackoverflow.com/questions/18716780/storm-v0-8-2-rebalance-command-not-updating-the-number-of-executors-for-a-bolt
>
> which seems to indicate that the number of executors is bounded
> by the number of tasks, and if number of executors = number of
> tasks (which it is in my case), we can't increase this further.
>
> Is this right ? Is there a way we can increase the number of
> executors at run time without restarting the topology (along
> with increasing number of workers) ?
>
> Thanks
> Hemanth
>
>
>
Re: Rebalancing executors
Posted by Hemanth Yamijala <yh...@gmail.com>.
Hi,
Maybe I asked too soon.
http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/
clearly says that the number of tasks is fixed for lifetime of the
topology. Number of executors == number of tasks by default. Hence, if the
number of executors is already at maximum level, it cannot go further. In
the comments section, it further says: "Configuring multiple tasks per
executor (thread) gives you the flexibility to expand the topology through
the "storm rebalance" command in the future without taking the topology
offline. In other words it is not a performance reason."
So, it appears the expectation is to overprovision the number of tasks,
start with minimal number of executors, and then grow executors to achieve
parallelism as workload increases. Is this right ?
If yes, overall, this is somewhat operationally difficult than expanding on
need. Is this a valid use case for Storm to support ? Any plans for this in
future ?
Thanks
hemanth
On Mon, Jun 22, 2015 at 5:34 PM, John Yost <so...@gmail.com>
wrote:
> This is an excellent question, need this info for myself as well.
>
> --John
>
> On Mon, Jun 22, 2015 at 7:40 AM, Hemanth Yamijala <yh...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I was testing the rebalance functionality on Storm 0.9.4.
>>
>> storm rebalance <name> -w 10 -n 2
>> - Works as expected. It increased the number of workers to 2.
>>
>> storm rebalance <name> -w 10 -n 2 -e <bolt-name>=20
>> - Works only for increasing the number of workers, but did *not* change
>> the number of executors of <bolt-name>.
>>
>> I came across this question on StackOverflow:
>> http://stackoverflow.com/questions/18716780/storm-v0-8-2-rebalance-command-not-updating-the-number-of-executors-for-a-bolt
>>
>>
>> which seems to indicate that the number of executors is bounded by the
>> number of tasks, and if number of executors = number of tasks (which it is
>> in my case), we can't increase this further.
>>
>> Is this right ? Is there a way we can increase the number of executors at
>> run time without restarting the topology (along with increasing number of
>> workers) ?
>>
>> Thanks
>> Hemanth
>>
>
>
Re: Rebalancing executors
Posted by John Yost <so...@gmail.com>.
This is an excellent question, need this info for myself as well.
--John
On Mon, Jun 22, 2015 at 7:40 AM, Hemanth Yamijala <yh...@gmail.com>
wrote:
> Hi,
>
> I was testing the rebalance functionality on Storm 0.9.4.
>
> storm rebalance <name> -w 10 -n 2
> - Works as expected. It increased the number of workers to 2.
>
> storm rebalance <name> -w 10 -n 2 -e <bolt-name>=20
> - Works only for increasing the number of workers, but did *not* change
> the number of executors of <bolt-name>.
>
> I came across this question on StackOverflow:
> http://stackoverflow.com/questions/18716780/storm-v0-8-2-rebalance-command-not-updating-the-number-of-executors-for-a-bolt
>
>
> which seems to indicate that the number of executors is bounded by the
> number of tasks, and if number of executors = number of tasks (which it is
> in my case), we can't increase this further.
>
> Is this right ? Is there a way we can increase the number of executors at
> run time without restarting the topology (along with increasing number of
> workers) ?
>
> Thanks
> Hemanth
>