[GitHub] [ozone] adoroszlai commented on a diff in pull request #3977: HDDS-7187. EC: Retry failed writes before rewrite to a new block group

[GitBox <gi...@apache.org>]

adoroszlai commented on code in PR #3977:
URL: https://github.com/apache/ozone/pull/3977#discussion_r1026408650


##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/ECXceiverClientGrpc.java:
##########
@@ -57,4 +79,59 @@ protected boolean shouldBlockAndWaitAsyncReply(
       ContainerProtos.ContainerCommandRequestProto request) {
     return false;
   }
+
+  @Override
+  protected ManagedChannel createChannel(DatanodeDetails dn, int port)
+      throws IOException {
+    NettyChannelBuilder channelBuilder =
+        NettyChannelBuilder.forAddress(dn.getIpAddress(), port).usePlaintext()
+            .maxInboundMessageSize(OzoneConsts.OZONE_SCM_CHUNK_MAX_SIZE)
+            .intercept(new GrpcClientInterceptor());
+    if (getSecConfig().isGrpcTlsEnabled()) {
+      SslContextBuilder sslContextBuilder = GrpcSslContexts.forClient();
+      if (getCaCerts() != null) {
+        sslContextBuilder.trustManager(getCaCerts());
+      }
+      if (getSecConfig().useTestCert()) {
+        channelBuilder.overrideAuthority("localhost");
+      }
+      channelBuilder.useTransportSecurity().
+          sslContext(sslContextBuilder.build());
+    } else {
+      channelBuilder.usePlaintext();
+    }
+    if (enableRetries) {
+      double maxAttempts = getConfig().getInt(OZONE_CLIENT_EC_GRPC_MAX_RETRIES,
+          OZONE_CLIENT_EC_GRPC_MAX_RETRIES_DEFAULT);
+
+      channelBuilder.defaultServiceConfig(createRetryServiceConfig(maxAttempts))
+          .maxRetryAttempts((int) maxAttempts).enableRetry();
+    }
+    return channelBuilder.build();

Review Comment:
   You could avoid duplicating most of channel creation logic by returning the `channelBuilder` instead.  The EC-specific override could simply call `super.createChannel(...)` and apply the retry-related further customisation to the builder.



##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -3275,6 +3275,24 @@
     </description>
   </property>
 
+  <property>
+    <name>ozone.client.ec.enable.retries</name>
+    <value>true</value>
+    <tag>CLIENT</tag>
+    <description>
+      To enable Grpc client retries for EC.
+    </description>
+  </property>
+
+  <property>
+    <name>ozone.client.ec.grpc.max.retries</name>

Review Comment:
   I'd prefer
   
    * `ozone.client.ec.grpc.retries.enabled`
    * `ozone.client.ec.grpc.retries.max`
   
   to keep these more closely related.  (Please feel free to drop `grpc` from both if you wish.)



##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientGrpc.java:
##########
@@ -605,6 +610,18 @@ public HddsProtos.ReplicationType getPipelineType() {
     return HddsProtos.ReplicationType.STAND_ALONE;
   }
 
+  public SecurityConfig getSecConfig() {
+    return secConfig;
+  }
+
+  public List<X509Certificate> getCaCerts() {
+    return caCerts;
+  }
+
+  public ConfigurationSource getConfig() {
+    return config;
+  }

Review Comment:
   These become unnecessary with the small refactoring.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org