You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by so...@apache.org on 2021/03/22 12:54:50 UTC
[hadoop] branch branch-3.3 updated (bc2bdee -> d8314ff)
This is an automated email from the ASF dual-hosted git repository.
sodonnell pushed a change to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from bc2bdee HDFS-15906. Close FSImage and FSNamesystem after formatting is complete (#2788)
new 793ab9a HDFS-15439. Setting dfs.mover.retry.max.attempts to negative value will retry forever. Contributed by AMC-team.
new d8314ff HDFS-15443. Setting dfs.datanode.max.transfer.threads to a very small value can cause strange failure. Contributed by AMC-team.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/hadoop/hdfs/server/datanode/DataXceiverServer.java | 3 +++
.../main/java/org/apache/hadoop/hdfs/server/mover/Mover.java | 10 +++++++++-
2 files changed, 12 insertions(+), 1 deletion(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/02: HDFS-15439. Setting dfs.mover.retry.max.attempts to
negative value will retry forever. Contributed by AMC-team.
Posted by so...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sodonnell pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 793ab9acf3c97bcb1fb3f25670a823cc05d5d9c4
Author: Ayush Saxena <ay...@apache.org>
AuthorDate: Sat Aug 15 15:07:08 2020 +0530
HDFS-15439. Setting dfs.mover.retry.max.attempts to negative value will retry forever. Contributed by AMC-team.
(cherry picked from commit e3d1966f58ad473b8e852aa2b11c8ed2b434d9e4)
---
.../main/java/org/apache/hadoop/hdfs/server/mover/Mover.java | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
index 2b36960..cae6b68 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
@@ -134,9 +134,17 @@ public class Mover {
final int maxNoMoveInterval = conf.getInt(
DFSConfigKeys.DFS_MOVER_MAX_NO_MOVE_INTERVAL_KEY,
DFSConfigKeys.DFS_MOVER_MAX_NO_MOVE_INTERVAL_DEFAULT);
- this.retryMaxAttempts = conf.getInt(
+ final int maxAttempts = conf.getInt(
DFSConfigKeys.DFS_MOVER_RETRY_MAX_ATTEMPTS_KEY,
DFSConfigKeys.DFS_MOVER_RETRY_MAX_ATTEMPTS_DEFAULT);
+ if (maxAttempts >= 0) {
+ this.retryMaxAttempts = maxAttempts;
+ } else {
+ LOG.warn(DFSConfigKeys.DFS_MOVER_RETRY_MAX_ATTEMPTS_KEY + " is "
+ + "configured with a negative value, using default value of "
+ + DFSConfigKeys.DFS_MOVER_RETRY_MAX_ATTEMPTS_DEFAULT);
+ this.retryMaxAttempts = DFSConfigKeys.DFS_MOVER_RETRY_MAX_ATTEMPTS_DEFAULT;
+ }
this.retryCount = retryCount;
this.dispatcher = new Dispatcher(nnc, Collections.<String> emptySet(),
Collections.<String> emptySet(), movedWinWidth, moverThreads, 0,
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/02: HDFS-15443. Setting
dfs.datanode.max.transfer.threads to a very small value can cause strange
failure. Contributed by AMC-team.
Posted by so...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sodonnell pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit d8314fffc0bbbb54f301f8cc4f4725b2fb84b9dc
Author: Ayush Saxena <ay...@apache.org>
AuthorDate: Sat Aug 8 14:33:53 2020 +0530
HDFS-15443. Setting dfs.datanode.max.transfer.threads to a very small value can cause strange failure. Contributed by AMC-team.
(cherry picked from commit c2a17659d1d48407edf437131783b653dbc87bc1)
---
.../java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java | 3 +++
1 file changed, 3 insertions(+)
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
index de5f415..4a017ed 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
@@ -188,6 +188,9 @@ class DataXceiverServer implements Runnable {
this.maxXceiverCount =
conf.getInt(DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_KEY,
DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_DEFAULT);
+ Preconditions.checkArgument(this.maxXceiverCount >= 1,
+ DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_KEY +
+ " should not be less than 1.");
this.estimateBlockSize = conf.getLongBytes(DFSConfigKeys.DFS_BLOCK_SIZE_KEY,
DFSConfigKeys.DFS_BLOCK_SIZE_DEFAULT);
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org