More Memory Usage with Storm 1.0.1 comparing with Storm 0.9.3

["Jeff Hu (BLOOMBERG/ 731 LEX)" <jh...@bloomberg.net>]

Comparing Apache Storm 0.10.0 and 1.0.1, we found significant performance improvements with 1.0.1 including less CPU usage during busy hours, latency, and GC.   However, we found that the memory size (VSIZE) is increased from 76G to 140G and resident set size(RSS) from 22G to 84G for a topology with the same yaml and the same memory setting of –Xms16G –Xmx16G.  

Can someone explain this?  Thanks

Re: More Memory Usage with Storm 1.0.1 comparing with Storm 0.9.3

[Abhishek Agarwal <ab...@gmail.com>]

It could be because internal queues are not directly bounded in 1.0.0. The
queue is size indirectly controlled through backpressure and
maxSpoutPending. So the queue population may be higher than previous version

On Jul 22, 2016 7:32 PM, "Jeff Hu (BLOOMBERG/ 731 LEX)" <jh...@bloomberg.net>
wrote:

Comparing Apache Storm 0.10.0 and 1.0.1, we found significant performance
improvements with 1.0.1 including less CPU usage during busy hours,
latency, and GC. However, we found that the memory size (VSIZE) is
increased from 76G to 140G and resident set size(RSS) from 22G to 84G for a
topology with the same yaml and the same memory setting of –Xms16G –Xmx16G.


Can someone explain this? Thanks