You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flume.apache.org by "Jain, Prem" <Pr...@netapp.com> on 2011/12/13 22:42:55 UTC

Flume transfer speeds

I am trying to ingest 1 TB of small files (40 - 200 MB) using flume into my 12 node HDFS cluster. I am getting very poor data transfer rates  of the order of 10 GB/Hr. I was expecting wire speeds on a GB network.
Is there anything I can do to speed this up ?

Thanks in advance