[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

["Karel Fuka (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044382#comment-16044382 ] 

Karel Fuka commented on PARQUET-783:
------------------------------------

Hi - is there any way we could help with this? It has become critical for us and obviously we understand there are many priorities, but if there is any way we could help to get this out, let us know.

What you be able to indicate at least what is the current release timeline?

Thanks

> H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks
> --------------------------------------------------------------------------------------------------
>
>                 Key: PARQUET-783
>                 URL: https://issues.apache.org/jira/browse/PARQUET-783
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>    Affects Versions: 1.9.0
>            Reporter: Michael Allman
>            Assignee: Michael Allman
>            Priority: Critical
>             Fix For: 1.10.0
>
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In the process, it opens a new {{FSDataInputStream}} and wraps it. However, {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is not closed. As a result, these stale connections can exhaust a clusters' data nodes' connection resources and lead to mysterious HDFS read failures in HDFS clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)