You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by John Pham <jo...@outlook.com> on 2015/03/01 09:41:17 UTC

Lower throughput with greater number of bolts

Hi all,
I'm fairly new to Storm/Trident and am having some trouble with performance tweaking a non-local Trident topology. For input, I am using Kafka with one million Kafka messages pre-loaded into the topic queue. I am forcing my topology to read from the start of the queue using the Kafka spout's forceFromStart configuration. I am testing these two topologies: One Kafka spout with one bolt that counts emitted tuples (using Storm's metrics consumer) and one bolt for general computation.One Kafka spout with one bolt that counts emitted tuples (using Storm's metrics consumer) and five bolts for general computation. The only difference between the first and second topologies are how many additional bolts I have for general computation. For the first one, I have measured just under 17000 tuples per second.For the second one, the measurement is three times slower at just under 5000 tuples per second. 
Is this expected behavior? Are there any specific guidelines about running a topology with a large number of bolts in this manner?I would like to be able to expand the number of general computation bolts to several hundred if that would be possible.
Thanks,John