You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/10/15 10:03:00 UTC
[jira] [Commented] (HDFS-17996) Make the regular file check in DirectoryScanner configurable
[ https://issues.apache.org/jira/browse/HDFS-17996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18127435#comment-18127435 ]
ASF GitHub Bot commented on HDFS-17996:
---------------------------------------
cxzl25 opened a new pull request, #8825:
URL: https://github.com/apache/hadoop/pull/8825
### Description of PR
HDFS-16316 added a check to DirectoryScanner#scan that verifies, for every
block, that both the block file and the meta file are regular files
(FileUtil#isRegularFile with NOFOLLOW_LINKS). The check runs during
reconciliation, after the reports of all volumes have been compiled, merged
and sorted by block id. It therefore issues two extra lstat calls per block
in block id order, which interleaves all volumes and sub-directories and has
no locality at all.
### How was this patch tested?
### For code changes:
- [ ] Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the endpoint
declared according to the connector-specific documentation? *Note: Automated CI
testing doesn't cover all cases so manual testing with cloud storage is still
required.*
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
### AI Tooling
If an AI tool was used:
- [ ] The PR includes the phrase "Contains content generated by <tool>"
where <tool> is the name of the AI tool used.
- [ ] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
> Make the regular file check in DirectoryScanner configurable
> ------------------------------------------------------------
>
> Key: HDFS-17996
> URL: https://issues.apache.org/jira/browse/HDFS-17996
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
> Reporter: dzcxzl
> Priority: Major
>
> HDFS-16316 added a check to DirectoryScanner#scan that verifies, for every
> block, that both the block file and the meta file are regular files
> (FileUtil#isRegularFile with NOFOLLOW_LINKS). The check runs during
> reconciliation, after the reports of all volumes have been compiled, merged
> and sorted by block id. It therefore issues two extra lstat calls per block
> in block id order, which interleaves all volumes and sub-directories and has
> no locality at all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org