You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/09/19 09:06:20 UTC
[GitHub] [ozone] symious opened a new pull request, #3762: HDDS-7234. Add a common option for DiskBalancer commands
symious opened a new pull request, #3762:
URL: https://github.com/apache/ozone/pull/3762
## What changes were proposed in this pull request?
This ticket is to add a common option for DiskBalancer CLI.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7234
## How was this patch tested?
original unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
symious commented on PR #3762:
URL: https://github.com/apache/ozone/pull/3762#issuecomment-1250756946
@lokeshj1703 Please help to review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 commented on a diff in pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on code in PR #3762:
URL: https://github.com/apache/ozone/pull/3762#discussion_r974986689
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerStartSubcommand.java:
##########
@@ -55,31 +54,23 @@ public class DiskBalancerStartSubcommand extends ScmSubcommand {
description = "Max parallelThread for DiskBalancer.")
private Optional<Integer> parallelThread;
- @Option(names = {"-a", "--allDatanodes"},
- description = "Start diskBalancer on all datanodes.")
- private boolean allHosts;
-
- @Parameters(description = "List of fully qualified host names")
- private List<String> hosts = new ArrayList<>();
+ @CommandLine.Mixin
+ private DiskBalancerCommonOptions commonOptions =
+ new DiskBalancerCommonOptions();
@Override
public void execute(ScmClient scmClient) throws IOException {
- if (hosts.size() == 0 && !allHosts) {
- System.out.println("Datanode not specified. Please specify " +
- "\"--allDatanodes\" to start diskBalancer on all datanodes");
- return;
- }
- if (hosts.size() != 0 && allHosts) {
- System.out.println("Confused options. Omit \"--allDatanodes\" or " +
- "Datanodes.");
+ if (!commonOptions.check()) {
return;
}
List<DatanodeAdminError> errors =
scmClient.startDiskBalancer(threshold, bandwidthInMB, parallelThread,
- hosts.size() == 0 ? Optional.empty() : Optional.of(hosts));
+ commonOptions.getHosts().size() == 0 ? Optional.empty() :
+ Optional.of(commonOptions.getHosts()));
Review Comment:
Create a function in DiskBalancerCommonOptions for this. We can reuse it in other classes as well.
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerCommonOptions.java:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.cli.datanode;
+
+import picocli.CommandLine;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Common options for DiskBalancer commands.
+ */
+public class DiskBalancerCommonOptions {
+
+ @CommandLine.Option(names = {"-a", "--all"},
+ description = "Run commands on all datanodes.")
+ private boolean allHosts;
+
+ @CommandLine.Option(names = {"-d", "--datanodes"},
+ description = "Run commands on specific datanodes.")
Review Comment:
Could we add some more info on how these datanodes should be specified? Like a comma separated list of datanodes ip or hostname. And we can provide an example as well.
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerStartSubcommand.java:
##########
@@ -55,31 +54,23 @@ public class DiskBalancerStartSubcommand extends ScmSubcommand {
description = "Max parallelThread for DiskBalancer.")
private Optional<Integer> parallelThread;
- @Option(names = {"-a", "--allDatanodes"},
- description = "Start diskBalancer on all datanodes.")
- private boolean allHosts;
-
- @Parameters(description = "List of fully qualified host names")
- private List<String> hosts = new ArrayList<>();
+ @CommandLine.Mixin
+ private DiskBalancerCommonOptions commonOptions =
+ new DiskBalancerCommonOptions();
@Override
public void execute(ScmClient scmClient) throws IOException {
- if (hosts.size() == 0 && !allHosts) {
- System.out.println("Datanode not specified. Please specify " +
- "\"--allDatanodes\" to start diskBalancer on all datanodes");
- return;
- }
- if (hosts.size() != 0 && allHosts) {
- System.out.println("Confused options. Omit \"--allDatanodes\" or " +
- "Datanodes.");
+ if (!commonOptions.check()) {
return;
}
List<DatanodeAdminError> errors =
scmClient.startDiskBalancer(threshold, bandwidthInMB, parallelThread,
- hosts.size() == 0 ? Optional.empty() : Optional.of(hosts));
+ commonOptions.getHosts().size() == 0 ? Optional.empty() :
+ Optional.of(commonOptions.getHosts()));
System.out.println("Start DiskBalancer on datanode(s):\n" +
- (allHosts ? "All datanodes" : String.join("\n", hosts)));
+ (commonOptions.isAllHosts() ? "All datanodes" :
+ String.join("\n", commonOptions.getHosts())));
Review Comment:
Create a function in DiskBalancerCommonOptions for this. We can reuse it in other classes as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ferhui merged pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
ferhui merged PR #3762:
URL: https://github.com/apache/ozone/pull/3762
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ferhui commented on pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
ferhui commented on PR #3762:
URL: https://github.com/apache/ozone/pull/3762#issuecomment-1253558730
@dineshchitlangia @lokeshj1703 Hi, do you have other comments? Plan to merge it tmr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] dineshchitlangia commented on a diff in pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
dineshchitlangia commented on code in PR #3762:
URL: https://github.com/apache/ozone/pull/3762#discussion_r974849560
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerCommonOptions.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.cli.datanode;
+
+import picocli.CommandLine;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Common options for DiskBalancer commands.
+ */
+public class DiskBalancerCommonOptions {
+
+ @CommandLine.Option(names = {"-a", "--allDatanodes"},
+ description = "Run commands on all datanodes.")
+ private boolean allHosts;
+
+ @CommandLine.Option(names = {"--hosts"},
+ description = "Run commands on specific datanodes.")
+ private List<String> hosts = new ArrayList<>();
+
+ /**
+ * Check the common options of DiskBalancerCommand.
+ * @return if the check passed
+ */
+ public boolean check() {
+ if (hosts.size() == 0 && !allHosts) {
+ System.out.println("Datanode not specified. Please specify " +
+ "\"--allDatanodes\" to start diskBalancer on all datanodes");
+ return false;
+ }
+ if (hosts.size() != 0 && allHosts) {
+ System.out.println("Confused options. Omit \"-a(--allDatanodes)\" or " +
+ "--hosts.");
Review Comment:
```suggestion
System.out.println("Invalid option selection. Use either \"-a(--allDatanodes)\" or " +
"--hosts.");
```
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerCommonOptions.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.cli.datanode;
+
+import picocli.CommandLine;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Common options for DiskBalancer commands.
+ */
+public class DiskBalancerCommonOptions {
+
+ @CommandLine.Option(names = {"-a", "--allDatanodes"},
+ description = "Run commands on all datanodes.")
+ private boolean allHosts;
+
+ @CommandLine.Option(names = {"--hosts"},
+ description = "Run commands on specific datanodes.")
+ private List<String> hosts = new ArrayList<>();
+
+ /**
+ * Check the common options of DiskBalancerCommand.
+ * @return if the check passed
+ */
+ public boolean check() {
+ if (hosts.size() == 0 && !allHosts) {
+ System.out.println("Datanode not specified. Please specify " +
+ "\"--allDatanodes\" to start diskBalancer on all datanodes");
Review Comment:
```suggestion
System.out.println("Datanode not specified. Please specify at least one datanode or use " +
"\"--allDatanodes\" to start diskBalancer on all datanodes");
```
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerCommonOptions.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.cli.datanode;
+
+import picocli.CommandLine;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Common options for DiskBalancer commands.
+ */
+public class DiskBalancerCommonOptions {
+
+ @CommandLine.Option(names = {"-a", "--allDatanodes"},
+ description = "Run commands on all datanodes.")
+ private boolean allHosts;
+
+ @CommandLine.Option(names = {"--hosts"},
+ description = "Run commands on specific datanodes.")
Review Comment:
To maintain consistency with other options, we should have both short and a long option name -> `-h` and `--hosts`
If `-h` is used for help or conflicts with any existing options, consider using `-d` and `--datanodes`
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerCommonOptions.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.cli.datanode;
+
+import picocli.CommandLine;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Common options for DiskBalancer commands.
+ */
+public class DiskBalancerCommonOptions {
+
+ @CommandLine.Option(names = {"-a", "--allDatanodes"},
+ description = "Run commands on all datanodes.")
Review Comment:
Consider replacing `--allDatanodes` with just `--all`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 commented on a diff in pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on code in PR #3762:
URL: https://github.com/apache/ozone/pull/3762#discussion_r976275349
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerStartSubcommand.java:
##########
@@ -55,31 +54,23 @@ public class DiskBalancerStartSubcommand extends ScmSubcommand {
description = "Max parallelThread for DiskBalancer.")
private Optional<Integer> parallelThread;
- @Option(names = {"-a", "--allDatanodes"},
- description = "Start diskBalancer on all datanodes.")
- private boolean allHosts;
-
- @Parameters(description = "List of fully qualified host names")
- private List<String> hosts = new ArrayList<>();
+ @CommandLine.Mixin
+ private DiskBalancerCommonOptions commonOptions =
+ new DiskBalancerCommonOptions();
@Override
public void execute(ScmClient scmClient) throws IOException {
- if (hosts.size() == 0 && !allHosts) {
- System.out.println("Datanode not specified. Please specify " +
- "\"--allDatanodes\" to start diskBalancer on all datanodes");
- return;
- }
- if (hosts.size() != 0 && allHosts) {
- System.out.println("Confused options. Omit \"--allDatanodes\" or " +
- "Datanodes.");
+ if (!commonOptions.check()) {
return;
}
List<DatanodeAdminError> errors =
scmClient.startDiskBalancer(threshold, bandwidthInMB, parallelThread,
- hosts.size() == 0 ? Optional.empty() : Optional.of(hosts));
+ commonOptions.getHosts().size() == 0 ? Optional.empty() :
+ Optional.of(commonOptions.getHosts()));
Review Comment:
@symious Can you please address this as well? I am +1 on the PR o.w.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
symious commented on PR #3762:
URL: https://github.com/apache/ozone/pull/3762#issuecomment-1252104699
@dineshchitlangia @lokeshj1703 Updated the PR, please have a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on a diff in pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
symious commented on code in PR #3762:
URL: https://github.com/apache/ozone/pull/3762#discussion_r976380734
##########
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerStartSubcommand.java:
##########
@@ -55,31 +54,23 @@ public class DiskBalancerStartSubcommand extends ScmSubcommand {
description = "Max parallelThread for DiskBalancer.")
private Optional<Integer> parallelThread;
- @Option(names = {"-a", "--allDatanodes"},
- description = "Start diskBalancer on all datanodes.")
- private boolean allHosts;
-
- @Parameters(description = "List of fully qualified host names")
- private List<String> hosts = new ArrayList<>();
+ @CommandLine.Mixin
+ private DiskBalancerCommonOptions commonOptions =
+ new DiskBalancerCommonOptions();
@Override
public void execute(ScmClient scmClient) throws IOException {
- if (hosts.size() == 0 && !allHosts) {
- System.out.println("Datanode not specified. Please specify " +
- "\"--allDatanodes\" to start diskBalancer on all datanodes");
- return;
- }
- if (hosts.size() != 0 && allHosts) {
- System.out.println("Confused options. Omit \"--allDatanodes\" or " +
- "Datanodes.");
+ if (!commonOptions.check()) {
return;
}
List<DatanodeAdminError> errors =
scmClient.startDiskBalancer(threshold, bandwidthInMB, parallelThread,
- hosts.size() == 0 ? Optional.empty() : Optional.of(hosts));
+ commonOptions.getHosts().size() == 0 ? Optional.empty() :
+ Optional.of(commonOptions.getHosts()));
Review Comment:
@lokeshj1703 Updated the patch, please have a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ferhui commented on pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
ferhui commented on PR #3762:
URL: https://github.com/apache/ozone/pull/3762#issuecomment-1254416970
@symious Thanks for your contribution! @dineshchitlangia @lokeshj1703 Thanks for your careful review! Merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #3762: HDDS-7234. Add a common option for DiskBalancer commands
Posted by GitBox <gi...@apache.org>.
symious commented on PR #3762:
URL: https://github.com/apache/ozone/pull/3762#issuecomment-1254418190
@ferhui @lokeshj1703 @dineshchitlangia Thank you for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org