[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

["Gabor Szadovszky (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721918#comment-15721918 ] 

Gabor Szadovszky commented on PARQUET-783:
------------------------------------------

Hi [~michael],
As you have linked the PR you should press the button "Submit Patch" so that the status of this jira will be highlighted for the committers.
I would also suggest assigning the JIRA for yourself.
Thanks a lot.

> H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks
> --------------------------------------------------------------------------------------------------
>
>                 Key: PARQUET-783
>                 URL: https://issues.apache.org/jira/browse/PARQUET-783
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>    Affects Versions: 1.9.0, 1.9.1
>            Reporter: Michael Allman
>            Priority: Critical
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In the process, it opens a new {{FSDataInputStream}} and wraps it. However, {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is not closed. As a result, these stale connections can exhaust a clusters' data nodes' connection resources and lead to mysterious HDFS read failures in HDFS clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)