You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by cn...@apache.org on 2015/02/18 21:03:29 UTC
[1/2] hadoop git commit: HDFS-7772. Document hdfs balancer
-exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao.
Repository: hadoop
Updated Branches:
refs/heads/branch-2 8f06eaa25 -> b1b1e05e1
refs/heads/trunk f5da5566d -> 2aa9979a7
HDFS-7772. Document hdfs balancer -exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/b1b1e05e
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/b1b1e05e
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/b1b1e05e
Branch: refs/heads/branch-2
Commit: b1b1e05e18c830ace9ad27d2c115196f79e8d0ad
Parents: 8f06eaa
Author: cnauroth <cn...@apache.org>
Authored: Wed Feb 18 11:47:45 2015 -0800
Committer: cnauroth <cn...@apache.org>
Committed: Wed Feb 18 12:03:00 2015 -0800
----------------------------------------------------------------------
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 3 +++
.../apache/hadoop/hdfs/server/balancer/Balancer.java | 7 +++----
.../hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm | 15 ++++++++++++++-
3 files changed, 20 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/b1b1e05e/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
index 436a605..3fb82c6 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
@@ -354,6 +354,9 @@ Release 2.7.0 - UNRELEASED
HDFS-7780. Update use of Iterator to Iterable in DataXceiverServer and
SnapshotDiffInfo. (Ray Chiang via aajisaka)
+ HDFS-7772. Document hdfs balancer -exclude/-include option in
+ HDFSCommands.html (Xiaoyu Yao via cnauroth)
+
OPTIMIZATIONS
HDFS-7454. Reduce memory footprint for AclEntries in NameNode.
http://git-wip-us.apache.org/repos/asf/hadoop/blob/b1b1e05e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
index 1075861..b429abc 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
@@ -171,15 +171,14 @@ public class Balancer {
private static final long GB = 1L << 30; //1GB
private static final long MAX_SIZE_TO_MOVE = 10*GB;
- private static final String USAGE = "Usage: java "
- + Balancer.class.getSimpleName()
+ private static final String USAGE = "Usage: hdfs balancer"
+ "\n\t[-policy <policy>]\tthe balancing policy: "
+ BalancingPolicy.Node.INSTANCE.getName() + " or "
+ BalancingPolicy.Pool.INSTANCE.getName()
+ "\n\t[-threshold <threshold>]\tPercentage of disk capacity"
- + "\n\t[-exclude [-f <hosts-file> | comma-sperated list of hosts]]"
+ + "\n\t[-exclude [-f <hosts-file> | <comma-separated list of hosts>]]"
+ "\tExcludes the specified datanodes."
- + "\n\t[-include [-f <hosts-file> | comma-sperated list of hosts]]"
+ + "\n\t[-include [-f <hosts-file> | <comma-separated list of hosts>]]"
+ "\tIncludes only the specified datanodes."
+ "\n\t[-idleiterations <idleiterations>]"
+ "\tNumber of consecutive idle iterations (-1 for Infinite) before exit.";
http://git-wip-us.apache.org/repos/asf/hadoop/blob/b1b1e05e/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm b/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
index aa7c7a7..de76c9d 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
@@ -134,7 +134,14 @@ HDFS Commands Guide
to stop the rebalancing process. See
{{{./HdfsUserGuide.html#Balancer}Balancer}} for more details.
- Usage: <<<hdfs balancer [-threshold <threshold>] [-policy <policy>] [-idleiterations <idleiterations>]>>>
++------------------------------------------+
+ Usage: hdfs balancer
+ [-threshold <threshold>]
+ [-policy <policy>]
+ [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
+ [-include [-f <hosts-file> | <comma-separated list of hosts>]]
+ [-idleiterations <idleiterations>]
++------------------------------------------+
*------------------------+----------------------------------------------------+
|| COMMAND_OPTION | Description
@@ -147,6 +154,12 @@ HDFS Commands Guide
| | <<<blockpool>>>: Cluster is balanced if each block
| | pool in each datanode is balanced.
*------------------------+----------------------------------------------------+
+| -exclude -f \<hosts-file\> \| | Excludes the specified datanodes from
+| \<comma-separated list of hosts\> | being balanced by the balancer.
+*------------------------+----------------------------------------------------+
+| -include -f \<hosts-file\> \| | Includes only the specified datanode
+| \<comma-separated list of hosts\> | to be balanced by the balancer.
+*------------------------+----------------------------------------------------+
| -idleiterations <iterations> | Maximum number of idle iterations before exit.
| | This overwrites the default idleiterations(5).
*------------------------+----------------------------------------------------+
[2/2] hadoop git commit: HDFS-7772. Document hdfs balancer
-exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao.
Posted by cn...@apache.org.
HDFS-7772. Document hdfs balancer -exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/2aa9979a
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/2aa9979a
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/2aa9979a
Branch: refs/heads/trunk
Commit: 2aa9979a713ab79853885264ad7739c48226aaa4
Parents: f5da556
Author: cnauroth <cn...@apache.org>
Authored: Wed Feb 18 11:46:57 2015 -0800
Committer: cnauroth <cn...@apache.org>
Committed: Wed Feb 18 12:03:07 2015 -0800
----------------------------------------------------------------------
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 3 +++
.../hadoop/hdfs/server/balancer/Balancer.java | 7 +++----
.../hadoop-hdfs/src/site/markdown/HDFSCommands.md | 17 +++++++++++++----
3 files changed, 19 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/2aa9979a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
index ec1c837..70eae1c 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
@@ -653,6 +653,9 @@ Release 2.7.0 - UNRELEASED
HDFS-7804. correct the haadmin command usage in #HDFSHighAvailabilityWithQJM.html
(Brahma Reddy Battula via umamahesh)
+ HDFS-7772. Document hdfs balancer -exclude/-include option in
+ HDFSCommands.html (Xiaoyu Yao via cnauroth)
+
OPTIMIZATIONS
HDFS-7454. Reduce memory footprint for AclEntries in NameNode.
http://git-wip-us.apache.org/repos/asf/hadoop/blob/2aa9979a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
index 5b87cb5..71338e6 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java
@@ -170,15 +170,14 @@ public class Balancer {
private static final long GB = 1L << 30; //1GB
private static final long MAX_SIZE_TO_MOVE = 10*GB;
- private static final String USAGE = "Usage: java "
- + Balancer.class.getSimpleName()
+ private static final String USAGE = "Usage: hdfs balancer"
+ "\n\t[-policy <policy>]\tthe balancing policy: "
+ BalancingPolicy.Node.INSTANCE.getName() + " or "
+ BalancingPolicy.Pool.INSTANCE.getName()
+ "\n\t[-threshold <threshold>]\tPercentage of disk capacity"
- + "\n\t[-exclude [-f <hosts-file> | comma-sperated list of hosts]]"
+ + "\n\t[-exclude [-f <hosts-file> | <comma-separated list of hosts>]]"
+ "\tExcludes the specified datanodes."
- + "\n\t[-include [-f <hosts-file> | comma-sperated list of hosts]]"
+ + "\n\t[-include [-f <hosts-file> | <comma-separated list of hosts>]]"
+ "\tIncludes only the specified datanodes."
+ "\n\t[-idleiterations <idleiterations>]"
+ "\tNumber of consecutive idle iterations (-1 for Infinite) before exit.";
http://git-wip-us.apache.org/repos/asf/hadoop/blob/2aa9979a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
index 6a7f34c..0573158 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
@@ -245,13 +245,22 @@ Commands useful for administrators of a hadoop cluster.
### `balancer`
-Usage: `hdfs balancer [-threshold <threshold>] [-policy <policy>] [-idleiterations <idleiterations>]`
+Usage:
+
+ hdfs balancer
+ [-threshold <threshold>]
+ [-policy <policy>]
+ [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
+ [-include [-f <hosts-file> | <comma-separated list of hosts>]]
+ [-idleiterations <idleiterations>]
| COMMAND\_OPTION | Description |
|:---- |:---- |
-| `-policy` *policy* | `datanode` (default): Cluster is balanced if each datanode is balanced.<br/> `blockpool`: Cluster is balanced if each block pool in each datanode is balanced. |
-| `-threshold` *threshold* | Percentage of disk capacity. This overwrites the default threshold. |
-| `-idleiterations` *iterations* | Maximum number of idle iterations before exit. This overwrites the default idleiterations(5). |
+| `-policy` \<policy\> | `datanode` (default): Cluster is balanced if each datanode is balanced.<br/> `blockpool`: Cluster is balanced if each block pool in each datanode is balanced. |
+| `-threshold` \<threshold\> | Percentage of disk capacity. This overwrites the default threshold. |
+| `-exclude -f` \<hosts-file\> \| \<comma-separated list of hosts\> | Excludes the specified datanodes from being balanced by the balancer. |
+| `-include -f` \<hosts-file\> \| \<comma-separated list of hosts\> | Includes only the specified datanodes to be balanced by the balancer. |
+| `-idleiterations` \<iterations\> | Maximum number of idle iterations before exit. This overwrites the default idleiterations(5). |
Runs a cluster balancing utility. An administrator can simply press Ctrl-C to stop the rebalancing process. See [Balancer](./HdfsUserGuide.html#Balancer) for more details.