You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/01/11 00:04:14 UTC
[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
bharatviswa504 opened a new pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436
…raft client.
## What changes were proposed in this pull request?
Add a config to tune the config value of "raft.client.async.outstanding-requests.max" config in raft client.
There is a property scm.container.client.max.outstanding.requests but this is used to set outStandingAppendsMax, but the config of this is mentioned as the "Controls the maximum number of outstanding async requests that can be handled by the Standalone as well as Ratis client". So, I removed this from ScmClientConfig, and created a new class for this property with prefix dfs.ratis.client to be consistent with other ratis client properties. And used the same for outStandingAppendsMax as before, not sure this is the correct way, but to keep same with current code, used it.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2875
Please replace this section with the link to the Apache JIRA)
## How was this patch tested?
UT and acceptance test run.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#discussion_r366619980
##########
File path: hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
##########
@@ -345,10 +337,30 @@ public void setMaxSize(int maxSize) {
this.maxSize = maxSize;
}
+ }
+
+ /**
+ * Configuration for ratis client.
+ */
+ @ConfigGroup(prefix = "dfs.ratis.client")
+ public class DFSRatisClientConfig {
+
+ @Config(key = "async.max.outstanding.requests",
Review comment:
With new-style Java config, we don't need to add, it generates an ozone-default-generated.xml. In that file, I can see this new config.
```
[root@ozone-test-1 ex]# cat ozone-default-generated.xml | grep -A4 "dfs.ratis.client.async"
<name>dfs.ratis.client.async.max.outstanding.requests</name>
<value>64</value>
<description>Controls the maximum number of outstanding async requests that can be handled by the Standalone as well as Ratis client.</description>
<tag>OZONE, CLIENT, PERFORMANCE</tag>
</property>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#discussion_r365700327
##########
File path: hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
##########
@@ -345,10 +337,30 @@ public void setMaxSize(int maxSize) {
this.maxSize = maxSize;
}
+ }
+
+ /**
+ * Configuration for ratis client.
+ */
+ @ConfigGroup(prefix = "dfs.ratis.client")
+ public class DFSRatisClientConfig {
+
+ @Config(key = "async.max.outstanding.requests",
Review comment:
I dont think we have a corresponding config in ozone-default.xml.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#discussion_r366620119
##########
File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java
##########
@@ -206,6 +206,10 @@ static RaftClient newRaftClient(RpcType rpcType, RaftPeerId leader,
GrpcConfigKeys.OutputStream.setOutstandingAppendsMax(properties,
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] lokeshj1703 merged pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
lokeshj1703 merged pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] lokeshj1703 commented on issue #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on issue #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#issuecomment-575201827
@bharatviswa504 Thanks for updating the PR! The changes look good to me. I have committed it to master branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#discussion_r366619222
##########
File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java
##########
@@ -206,6 +206,10 @@ static RaftClient newRaftClient(RpcType rpcType, RaftPeerId leader,
GrpcConfigKeys.OutputStream.setOutstandingAppendsMax(properties,
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#issuecomment-574416321
@lokeshj1703 Thank You for the review. Addressed review comments and acceptance test failures.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#discussion_r366619222
##########
File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java
##########
@@ -206,6 +206,10 @@ static RaftClient newRaftClient(RpcType rpcType, RaftPeerId leader,
GrpcConfigKeys.OutputStream.setOutstandingAppendsMax(properties,
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on a change in pull request #436: HDDS-2875. Add a config in ozone to tune max outstanding requests in …
URL: https://github.com/apache/hadoop-ozone/pull/436#discussion_r365699229
##########
File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java
##########
@@ -206,6 +206,10 @@ static RaftClient newRaftClient(RpcType rpcType, RaftPeerId leader,
GrpcConfigKeys.OutputStream.setOutstandingAppendsMax(properties,
Review comment:
I think we can remove this setter. This is used by GrpcOutputStream in ratis which is not used by ozone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org