You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-dev@hadoop.apache.org by lei liu <li...@gmail.com> on 2013/04/17 15:54:31 UTC

HDFS pread performance

I test the HDFS pread performance, the avg time of pread is about 10ms,
but  pread max time reach 200ms, there is about one percent of pread
time is 200ms, that result to my application is timeout.

  I find  max time of the
RemoteBlockReader<eclipse-javadoc:%E2%98%82=hadoop-0.20.2-cdh3u5/src%5C/hdfs%3Corg.apache.hadoop.hdfs%7BDFSClient.java%E2%98%83DFSClient%E2%98%83RemoteBlockReader>.readChunk
method also can reach 100ms.  The
RemoteBlockReader<eclipse-javadoc:%E2%98%82=hadoop-0.20.2-cdh3u5/src%5C/hdfs%3Corg.apache.hadoop.hdfs%7BDFSClient.java%E2%98%83DFSClient%E2%98%83RemoteBlockReader>.readChunk
only read 512 bytes form socket, so  I think I can tuning linux socket
Kernel parameters to reduce the max time of pread. If anybody has the
experience,
please tell me.


Thansk,

LiuLei