You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/04/22 00:06:00 UTC
[jira] [Commented] (HDFS-17824) DataNode fails to start when dfs.datanode.directoryscan.threads is 0 or negative
[ https://issues.apache.org/jira/browse/HDFS-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18075292#comment-18075292 ]
ASF GitHub Bot commented on HDFS-17824:
---------------------------------------
deepujain commented on PR #8309:
URL: https://github.com/apache/hadoop/pull/8309#issuecomment-4292641860
Rebased this onto current apache/trunk and force-pushed the refreshed branch. Local validation passed with JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home /opt/homebrew/bin/mvn -Dmaven.repo.local=/tmp/codex-m2 test -pl hadoop-hdfs-project/hadoop-hdfs -am -Dtest=TestDirectoryScanner#testInvalidDirectoryScanThreadsUsesDefault,TestBalancerWithHANameNodes -DskipTests=false (6 tests, 0 failures).
> DataNode fails to start when dfs.datanode.directoryscan.threads is 0 or negative
> --------------------------------------------------------------------------------
>
> Key: HDFS-17824
> URL: https://issues.apache.org/jira/browse/HDFS-17824
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.8.5
> Reporter: AMC-team
> Assignee: AMC-team
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-17824.000.patch
>
>
> When {{dfs.datanode.directoryscan.threads}} is configured to {{0}} or a negative value, the DataNode initialization path constructs the directory scanner with {{{}Executors.newFixedThreadPool(threads){}}}. Because the thread count is non-positive, the JDK throws {{{}java.lang.IllegalArgumentException{}}}, and the DataNode fails to start. The code currently does not validate this configuration before creating the thread pool.
> *Steps to Reproduce*
> 1. Set the properties above in {{{}hdfs-site.xml{}}}.
> 2. Start or restart the DataNode.
> 3. Observe the following stack trace and process exit:
> {code:java}
> java.lang.IllegalArgumentException
> at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1314)
> at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1237)
> at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:151)
> at org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.<init>(DirectoryScanner.java:* )
> at org.apache.hadoop.hdfs.server.datanode.DataNode.initDirectoryScanner(DataNode.java:* )
> at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1543)
> ...
> {code}
> *Actual Result*
> DataNode startup aborts with IllegalArgumentException during DirectoryScanner initialization; the node never becomes available.
> *Expected Result*
> Non-positive values of {{dfs.datanode.directoryscan.threads}} are handled gracefully—either rejected early with a clear configuration error naming the key and offending value, or automatically clamped to a safe default (≥1) with a {{{}WARN{}}}, rather than crashing the process.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org