You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Max Xie (Jira)" <ji...@apache.org> on 2021/08/19 09:59:00 UTC
[jira] [Work started] (HDFS-16180) FsVolumeImpl.nextBlock should
consider that the block meta file has been deleted.
[ https://issues.apache.org/jira/browse/HDFS-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HDFS-16180 started by Max Xie.
---------------------------------------
> FsVolumeImpl.nextBlock should consider that the block meta file has been deleted.
> ---------------------------------------------------------------------------------
>
> Key: HDFS-16180
> URL: https://issues.apache.org/jira/browse/HDFS-16180
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.3.0, 3.4.0
> Reporter: Max Xie
> Assignee: Max Xie
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In my cluster, we found that when VolumeScanner run, sometime dn will throw some error log below
> ```
>
> 2021-08-19 08:00:11,549 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Deleted BP-1020175758-nnip-1597745872895 blk_1142977964_69237147 URI file:/disk1/dfs/data/current/BP-1020175758- nnip-1597745872895/current/finalized/subdir0/subdir21/blk_1142977964
> 2021-08-19 08:00:48,368 ERROR org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl: nextBlock(DS-060c8e4c-1ef6-49f5-91ef-91957356891a, BP-1020175758- nnip-1597745872895): I/O error
> java.io.IOException: Meta file not found, blockFile=/disk1/dfs/data/current/BP-1020175758- nnip-1597745872895/current/finalized/subdir0/subdir21/blk_1142977964
> at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetUtil.findMetaFile(FsDatasetUtil.java:101)
> at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl$BlockIteratorImpl.nextBlock(FsVolumeImpl.java:809)
> at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:528)
> at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:628)
> 2021-08-19 08:00:48,368 WARN org.apache.hadoop.hdfs.server.datanode.VolumeScanner: VolumeScanner(/disk1/dfs/data, DS-060c8e4c-1ef6-49f5-91ef-91957356891a): nextBlock error on org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl$BlockIteratorImpl@7febc6b4
> ```
> When VolumeScanner scan block blk_1142977964, it has been deleted by datanode, scanner can not find the meta file of blk_1142977964, so it throw these error log.
>
> Maybe we should handle FileNotFoundException during nextblock to reduce error log and nextblock retry times.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org