You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/09/08 23:40:00 UTC
[jira] [Commented] (HDFS-17978) Ensure requestShortCircuitFdsForRead() closes the block stream when metadata-stream setup fails
[ https://issues.apache.org/jira/browse/HDFS-17978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18113055#comment-18113055 ]
ASF GitHub Bot commented on HDFS-17978:
---------------------------------------
iamsanjaymalakar opened a new pull request, #8731:
URL: https://github.com/apache/hadoop/pull/8731
### Problem
`org.apache.hadoop.hdfs.server.datanode.DataNode.requestShortCircuitFdsForRead()` opens the block data stream into `fis[0]` before opening the metadata stream into `fis[1]`. `org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.getMetaDataInputStream()` (and `checkStorageState` / `getBlockInputStream`) can throw `java.io.IOException`. The catch clause handles only `java.lang.ClassCastException`, so the `java.io.IOException` propagates with `fis[0]` already open. The `fis` array is never returned on that path, so `fis[0]` is leaked.
### Change
* Add a `java.io.IOException` catch that closes any stream already stored in `fis` via `org.apache.hadoop.io.IOUtils.cleanupWithLogger` before rethrowing the original exception.
* The successful path is unchanged.
### Testing
* `mvn -pl hadoop-hdfs-project/hadoop-hdfs -am -Dtest=TestDataNodeShortCircuitFds test`
* New test `TestDataNodeShortCircuitFds#testClosesBlockStreamWhenMetadataStreamFails` fails against the previous code and passes with the change.
> Ensure requestShortCircuitFdsForRead() closes the block stream when metadata-stream setup fails
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-17978
> URL: https://issues.apache.org/jira/browse/HDFS-17978
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 3.4.1
> Reporter: Sanjay Malakar
> Priority: Major
>
> {\{org.apache.hadoop.hdfs.server.datanode.DataNode.requestShortCircuitFdsForRead()}} opens the block data stream into \{{fis[0]}} before opening the metadata stream into \{{fis[1]}}. \{{DatanodeUtil.getMetaDataInputStream()}} (and \{{checkStorageState}} / \{{getBlockInputStream}}) can throw \{{{}java.io.IOException{}}}. The catch clause handles only \{{java.lang.ClassCastException}}, so the \{{java.io.IOException}} propagates with \{{fis[0]}} already open. The \{{fis}} array is never returned on that path, so \{{fis[0]}} is leaked.
> {\{requestShortCircuitFdsForRead()}} should close any stream already stored in \{{fis}} before propagating the \{{java.io.IOException}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org