Resource consumption of the acker bolts?

[Navin Ipe <na...@searchlighthealth.com>]

Hi,

*The problem:*
A colleague says having 300 workers and 600 executors is a bad idea even if
300 of the 600 are acker bolts. Processing bottleneck, according to him.

*My question: *
So far I didn't experience a performance issue, but given the nature of the
application, number of tuples, processing and the number of emits can
become more demanding.
I can understand if a bottleneck is due to my number of bolts being greater
than the number of workers. But if my bolts = number of workers, then does
creating an equal number of extra ackers really consume so much memory and
processing power, that I would have to increase the number of workers to
double?

*The scenario:*
I have a single spout that emits around 20K tuples (I know it should be 1
per spout, but in this scenario it's a necessity). There are 300 bolts
receiving the tuples with fields grouping, and 300 ackers. 76 slots are
available.

On the Storm GUI, there are around 600 executors being shown, which I
reckon are my bolts + the acker bolts.

From Nathan Marz' answer
<https://groups.google.com/forum/#%21topic/storm-user/uJt-ym5xe60>, setting
the number of workers = number of ackers seems a safe bet. Also, he says a
bottleneck is application specific
<https://groups.google.com/forum/#%21topic/storm-user/kz_p2DITakE>, and
that he planned to improve the worker parallelism (wonder if it is
implemented yet).


-- 
Regards,
Navin