[jira] [Commented] (HBASE-11660) Make WAL reader follow contract for java.io.InputStream.available()

["Eric Hanson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084891#comment-14084891 ] 

Eric Hanson commented on HBASE-11660:
-------------------------------------

The existing WAL reading code appears to take a hard dependency on the behavior of the available method in DFSInputStream.java, which looks like this in the Hadoop trunk:
{code}
/** Return the size of the remaining available bytes
* if the size is less than or equal to {@link Integer#MAX_VALUE},
* otherwise, return {@link Integer#MAX_VALUE}.
*/
  @Override
  public synchronized int available() throws IOException {
    if (closed) {
      throw new IOException("Stream closed");
    }

    final long remaining = getFileLength() - pos;
    return remaining <= Integer.MAX_VALUE? (int)remaining: Integer.MAX_VALUE;
  }
{code}
This implied contract is different from the one in java.io.InputStream.

> Make WAL reader follow contract for java.io.InputStream.available()
> -------------------------------------------------------------------
>
>                 Key: HBASE-11660
>                 URL: https://issues.apache.org/jira/browse/HBASE-11660
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>            Reporter: Eric Hanson
>            Priority: Minor
>         Attachments: hbase-11660.01.patch
>
>
> In the process of building support to running HBase on Microsoft Azure HDInsight, I hit an issue in the HBase WAL reading process that took a lot of time to debug. The WAL reading code depends on available() for the log InputStream never returing 0 until end of file. This is not the same as the contract in java.io.InputStream for available. 
> To prevent future grief for others that may want to port HBase onto storage systems other than HDFS, I propose to change the HBase WAL reader so it does not assume that EOF has been reached when available() == 0. It instead would treat available only as described in InputStream, i.e. available() is merely the number of bytes that could be read from the stream without blocking. That could be 0 even before EOF.



--
This message was sent by Atlassian JIRA
(v6.2#6252)