You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/12/07 06:44:24 UTC
[GitHub] [ozone] siddhantsangwan opened a new pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
siddhantsangwan opened a new pull request #2892:
URL: https://github.com/apache/ozone/pull/2892
## What changes were proposed in this pull request?
Making the defaults smarter and more readable, such as expressing threshold in percentage, making `maxSizeEnteringTarget` equal the size of five containers etc.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5927
## How was this patch tested?
`TestContainerBalancer` unit tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#issuecomment-1000129663
A summary of the changes:
1. Deprecate the proto fields `idleiterations` and `maxDatanodesRatioToInvolvePerIteration` and refactor them to `iterations` and `maxDatanodesPercentageToInvolvePerIteration`.
2. Represent threshold as a percentage instead of a ratio. Since ratios are used in calculations, add a method `getThresholdAsRatio` in `ContainerBalancerConfiguration`.
3. Change the default values of configurations along with their javadocs and related methods.
4. This PR also pulls in changes from the related PR #2893 , so let's get it merged first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#issuecomment-1000168193
@lokeshj1703 @JacksonYao287 @errose28 please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
errose28 commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r768960152
##########
File path: hadoop-hdds/interface-admin/src/main/resources/proto.lock
##########
@@ -1334,7 +1334,7 @@
},
{
"id": 3,
- "name": "idleiterations",
Review comment:
Don't EVER change proto lock files! These are automatically updated by protolock only on each major or minor release to ensure compatibility. This is an easy mistake to make when proto-backwards-compatability-check fails though, and it would definitely be helpful if our CI could help the devs out here : ) @adoroszlai is there an easy check we can put in CI to fail PRs that that modify anything called proto.lock?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r774515869
##########
File path: hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto
##########
@@ -479,11 +479,14 @@ message GetContainerTokenResponseProto {
message StartContainerBalancerRequestProto {
optional string traceID = 1;
optional double threshold = 2;
- optional int32 idleiterations = 3;
- optional double maxDatanodesRatioToInvolvePerIteration = 4;
+ optional int32 idleiterations = 3 [deprecated = true];
+ optional double maxDatanodesRatioToInvolvePerIteration = 4 [deprecated =
Review comment:
can we just replace them with the new. Compatibility may be not considered for now
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerConfiguration.java
##########
@@ -47,68 +41,66 @@
public final class ContainerBalancerConfiguration {
private static final Logger LOG =
LoggerFactory.getLogger(ContainerBalancerConfiguration.class);
- private OzoneConfiguration ozoneConfiguration;
@Config(key = "utilization.threshold", type = ConfigType.AUTO, defaultValue =
- "0.1", tags = {ConfigTag.BALANCER},
- description = "Threshold is a fraction in the range of 0 to 1. A " +
+ "10", tags = {ConfigTag.BALANCER},
+ description = "Threshold is a percentage in the range of 0 to 100. A " +
"cluster is considered balanced if for each datanode, the " +
"utilization of the datanode (used space to capacity ratio) differs" +
" from the utilization of the cluster (used space to capacity ratio" +
- " of the entire cluster) no more than the threshold value.")
- private String threshold = "0.1";
+ " of the entire cluster) no more than the threshold.")
+ private String threshold = "10";
Review comment:
can we also make threshold int not String?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r774515572
##########
File path: hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto
##########
@@ -479,11 +479,14 @@ message GetContainerTokenResponseProto {
message StartContainerBalancerRequestProto {
optional string traceID = 1;
optional double threshold = 2;
- optional int32 idleiterations = 3;
- optional double maxDatanodesRatioToInvolvePerIteration = 4;
+ optional int32 idleiterations = 3 [deprecated = true];
+ optional double maxDatanodesRatioToInvolvePerIteration = 4 [deprecated =
Review comment:
can we just replace them with the new. Compatibility may be not considered for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#issuecomment-1008888064
@lokeshj1703 Thanks for the review! I've updated the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 commented on pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#issuecomment-1010690428
@siddhantsangwan Thanks for the contribution! @errose28 @JacksonYao287 Thanks for the reviews! I have merged the PR to master branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#issuecomment-987638410
Hey @lokeshj1703 @JacksonYao287
We can discuss the balancer defaults here. I've made some initial changes:
1. Changed `idleIterations` to `iterations` to make it clear that this configuration means the number of iterations that balancer will run for.
2. Changed `maxSizeEnteringTarget` and `maxSizeLeavingSource` to allow the size of five containers, by default.
Proposed changes:
1. Change the definitions of configurations from ratio to percentage to make them more intuitive. For example, threshold can be "A percentage in the range of 0 to 100..."
2. Make the default for `maxSizeToMovePerIteration` something like `maxDatanodesRatioToInvolvePerIteration * number of healthy, in service DNs * maxSizeEnteringTarget`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r774674504
##########
File path: hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto
##########
@@ -479,11 +479,14 @@ message GetContainerTokenResponseProto {
message StartContainerBalancerRequestProto {
optional string traceID = 1;
optional double threshold = 2;
- optional int32 idleiterations = 3;
- optional double maxDatanodesRatioToInvolvePerIteration = 4;
+ optional int32 idleiterations = 3 [deprecated = true];
+ optional double maxDatanodesRatioToInvolvePerIteration = 4 [deprecated =
Review comment:
The backwards compatibility goal fails during build if I just replace them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
errose28 commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r769902700
##########
File path: hadoop-hdds/interface-admin/src/main/resources/proto.lock
##########
@@ -1334,7 +1334,7 @@
},
{
"id": 3,
- "name": "idleiterations",
Review comment:
Yeah the rename in the proto itself is safe, lets just make sure the proto lock stays in its original state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r781053806
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocolServerSideTranslatorPB.java
##########
@@ -718,22 +718,31 @@ public StartContainerBalancerResponseProto startContainerBalancer(
StartContainerBalancerRequestProto request)
throws IOException {
Optional<Double> threshold = Optional.empty();
- Optional<Integer> idleiterations = Optional.empty();
- Optional<Double> maxDatanodesRatioToInvolvePerIteration = Optional.empty();
+ Optional<Integer> iterations = Optional.empty();
+ Optional<Integer> maxDatanodesPercentageToInvolvePerIteration =
+ Optional.empty();
Optional<Long> maxSizeToMovePerIterationInGB = Optional.empty();
Optional<Long> maxSizeEnteringTargetInGB = Optional.empty();
Optional<Long> maxSizeLeavingSourceInGB = Optional.empty();
if(request.hasThreshold()) {
threshold = Optional.of(request.getThreshold());
}
- if(request.hasIdleiterations()) {
- idleiterations = Optional.of(request.getIdleiterations());
+
+ if (request.hasIterations()) {
Review comment:
Should we also handle idleIterations here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 merged pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
lokeshj1703 merged pull request #2892:
URL: https://github.com/apache/ozone/pull/2892
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r774674504
##########
File path: hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto
##########
@@ -479,11 +479,14 @@ message GetContainerTokenResponseProto {
message StartContainerBalancerRequestProto {
optional string traceID = 1;
optional double threshold = 2;
- optional int32 idleiterations = 3;
- optional double maxDatanodesRatioToInvolvePerIteration = 4;
+ optional int32 idleiterations = 3 [deprecated = true];
+ optional double maxDatanodesRatioToInvolvePerIteration = 4 [deprecated =
Review comment:
The backwards compatibility goal fails during compilation if I just replace them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan edited a comment on pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan edited a comment on pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#issuecomment-1000129663
A summary of the changes:
1. Deprecate the proto fields `idleiterations` and `maxDatanodesRatioToInvolvePerIteration` and refactor them to `iterations` and `maxDatanodesPercentageToInvolvePerIteration`.
2. Represent threshold as a percentage instead of a ratio. Since ratios are used in calculations, add a method `getThresholdAsRatio` in `ContainerBalancerConfiguration`.
3. Change the default values of configurations along with their javadocs and related methods.
4. This PR also pulls in changes from the related PR #2893 , so let's get that PR merged first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#issuecomment-1010686314
Thanks for the review @JacksonYao287 !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r769470436
##########
File path: hadoop-hdds/interface-admin/src/main/resources/proto.lock
##########
@@ -1334,7 +1334,7 @@
},
{
"id": 3,
- "name": "idleiterations",
Review comment:
Good catch. This seems to have sneaked in while I was refactoring this field's name across the project. CI got skipped since this is still a draft PR.
BTW, from what I understand, it's compatible to change the name of a proto field as long as the ID and type remain the same, and JSON is not involved. The [hadoop policy](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/Compatibility.html#Policy) that's linked [here](https://github.com/apache/ozone/blob/af4730a54bd2bbc63d556936292ae760701b7e7f/hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto#L21) also says it's compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r770461045
##########
File path: hadoop-hdds/interface-admin/src/main/resources/proto.lock
##########
@@ -1334,7 +1334,7 @@
},
{
"id": 3,
- "name": "idleiterations",
Review comment:
Changing only the proto file fails the backwards compatibility check here:
```
[INFO] CONFLICT: "StartContainerBalancerRequestProto" field: "idleiterations" has been removed, but is not reserved [ScmAdminProtocol.proto]
[INFO] CONFLICT: "StartContainerBalancerRequestProto" field: "iterations" ID: 3 has an updated name, previously "idleiterations" [ScmAdminProtocol.proto]
```
With the proto lock file in its original state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a change in pull request #2892: HDDS-5927. Improve defaults in ContainerBalancerConfiguration
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on a change in pull request #2892:
URL: https://github.com/apache/ozone/pull/2892#discussion_r774677115
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerConfiguration.java
##########
@@ -47,68 +41,66 @@
public final class ContainerBalancerConfiguration {
private static final Logger LOG =
LoggerFactory.getLogger(ContainerBalancerConfiguration.class);
- private OzoneConfiguration ozoneConfiguration;
@Config(key = "utilization.threshold", type = ConfigType.AUTO, defaultValue =
- "0.1", tags = {ConfigTag.BALANCER},
- description = "Threshold is a fraction in the range of 0 to 1. A " +
+ "10", tags = {ConfigTag.BALANCER},
+ description = "Threshold is a percentage in the range of 0 to 100. A " +
"cluster is considered balanced if for each datanode, the " +
"utilization of the datanode (used space to capacity ratio) differs" +
" from the utilization of the cluster (used space to capacity ratio" +
- " of the entire cluster) no more than the threshold value.")
- private String threshold = "0.1";
+ " of the entire cluster) no more than the threshold.")
+ private String threshold = "10";
Review comment:
Yes that's what I wanted to do. But we'll need to deprecate threshold in the proto message and add a new field with the int32 type. Should I go ahead with that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org