[jira] [Commented] (HDFS-6695) Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to implement read timeouts

["Colin Patrick McCabe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063879#comment-14063879 ] 

Colin Patrick McCabe commented on HDFS-6695:
--------------------------------------------

There are some docs here:  http://docs.oracle.com/javase/7/docs/api/java/nio/channels/AsynchronousFileChannel.html

It's a little unclear what the actual implementation of {{AsynchronousFileChannel}} is.  If it's just a thread pool, then it might not be high-performance enough to be useful here.  If it uses the underlying asynchronous I/O primitives provided by the operating system, then it would probably be useful for implementing these timeouts.

Another way we could do this on Linux is to read from a pipe instead of a file, and use the {{splice}} system call to move pages from a local disk file into the pipe in a high-performance fashion.  But I guess this doesn't work with position reads or alternative operating systems, so maybe it's not that useful....

> Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to implement read timeouts
> ----------------------------------------------------------------------------------------------
>
>                 Key: HDFS-6695
>                 URL: https://issues.apache.org/jira/browse/HDFS-6695
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Colin Patrick McCabe
>
> In BlockReaderLocal, the "read" system call could block for a long time if the disk drive is having problems, or there is a huge amount of I/O contention.  This might cause poor latency performance.
> In the remote block readers, we have implemented a read timeout, but we don't have one for the local block reader, since {{FileChannel#read}} doesn't support this.  
> Once we move to JDK 7, we should investigate the {{java.nio.file}} nonblocking file I/O package to see if it could be used to implement read timeouts.



--
This message was sent by Atlassian JIRA
(v6.2#6252)