You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by mahendran m <ma...@hotmail.com> on 2015/04/24 06:41:30 UTC

Question on flume performance tuning

Hi ,
I am consuming the message from ActiveMQ source using flume JMS source . and i am cousimg the message from two queue say SlowerQueue and FasterQueue
SlowerQueue Message incoming rate is 10m/s 
Now i am configure the flume agent with two different configuration
Configuration1	Java memory - 100MB -250MB        Source batch size- 50 event	capacity - 5000event	T.Capacity - 200event	HDFS batch size - 100event
Configuration2
	Java memory - 500MB -700MB
        Source batch size- 100 event	capacity - 10000event
	T.Capacity - 1000event
	HDFS batch size - 500eventQ: for Slower queue which configuration provides better performance result ?
FasterQueue Message incoming rate is 1000m/s 
Now i am configure the flume agent with two different configuration
Configuration1	Java memory - 100MB -250MB        Source batch size- 50 event	capacity - 5000event	T.Capacity - 200event	HDFS batch size - 100event
Configuration2
	Java memory - 500MB -700MB
        Source batch size- 100 event	capacity - 10000event
	T.Capacity - 1000event
	HDFS batch size - 500eventQ: for Faster queue which configuration provides better performance result ?
Regards,Mahendran