You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Martin Mituzas <xi...@hotmail.com> on 2010/02/05 17:03:48 UTC

Using two threads to read data from disk and send out in DataXceiver ?

hi, all
In current hdfs code, when DFSClient reads a file, DataXceiver will create a
thread to read data from its own disk and then transmit the data to
DFSClient via network.  I wonder if the performance will be improved by
using two different threads for reading disk and sending out seperately? Any
comments?
-- 
View this message in context: http://old.nabble.com/Using-two-threads-to-read-data-from-disk-and-send-out-in-DataXceiver---tp27469915p27469915.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: Using two threads to read data from disk and send out in DataXceiver ?

Posted by Todd Lipcon <to...@cloudera.com>.

Hi Martin,

Not sure what you mean - why would it be faster to split it into two
threads? Keep in mind that there is a TCP send buffer so if the client
is reading faster than the disk, the server's sends won't block
anyway.

-Todd

On Fri, Feb 5, 2010 at 8:03 AM, Martin Mituzas <xi...@hotmail.com> wrote:
>
> hi, all
> In current hdfs code, when DFSClient reads a file, DataXceiver will create a
> thread to read data from its own disk and then transmit the data to
> DFSClient via network.  I wonder if the performance will be improved by
> using two different threads for reading disk and sending out seperately? Any
> comments?
> --
> View this message in context: http://old.nabble.com/Using-two-threads-to-read-data-from-disk-and-send-out-in-DataXceiver---tp27469915p27469915.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>