You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Russole (via GitHub)" <gi...@apache.org> on 2026/04/14 16:26:49 UTC

[PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Russole opened a new pull request, #10079:
URL: https://github.com/apache/ozone/pull/10079

   ## What changes were proposed in this pull request?
   
   * Add a new `invokers` map in `SCMStateMachine` to use `ScmInvoker` for method dispatch
   * Replace reflection-based invocation with direct invocation via `ScmInvoker`
   * Add `getReturnType()` in `ScmInvoker` to avoid reflection when applying local state machine operations
   * Extend `ScmInvokerCodeGenerator` to generate the `getReturnType()` method for each invoker
   * Prevent checkstyle failures caused by generated invoker classes
   
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/HDDS-14974
   ## How was this patch tested?
   - All CI checks passed
      - https://github.com/Russole/ozone/actions/runs/24354244039
   - Verified that code generated by `ScmInvokerCodeGenerator` is functionally equivalent to the existing `DeletedBlockLogStateManagerInvoker`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on PR #10079:
URL: https://github.com/apache/ozone/pull/10079#issuecomment-4254101492

   Thanks @szetszwo and @sreejasahithi for the review.
   I have updated the patch based on the feedback.
   Further improvements to `ScmInvokerCodeGenerator` will be handled in a separate JIRA.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on PR #10079:
URL: https://github.com/apache/ozone/pull/10079#issuecomment-4263312298

   @szetszwo ,I'll take a look at the TestDeletedBlockLog failure.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3088254131


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/invoker/DeletedBlockLogStateManagerInvoker.java:
##########
@@ -115,7 +119,9 @@ public Object invokeLocal(String methodName, Object[] p) throws Exception {
       return null;

Review Comment:
   Good catch, thanks!
   Updated to return `Message.EMPTY`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on PR #10079:
URL: https://github.com/apache/ozone/pull/10079#issuecomment-4262516506

   Thanks @adoroszlai , @szetszwo  for catching this. I've updated the patch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "sreejasahithi (via GitHub)" <gi...@apache.org>.
sreejasahithi commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3084225443


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMStateMachine.java:
##########
@@ -200,6 +212,24 @@ public static Message process(final SCMRatisRequest request, Object handler) thr
     }
   }
 
+  public static Message process(final SCMRatisRequest request,
+      final ScmInvoker<?> invoker) throws Exception {
+
+    if (invoker == null) {
+      throw new IOException("No invoker found for request type "
+          + request.getType());
+    }
+
+    final Object result = invoker.invoke(
+        request.getOperation(),
+        request.getArguments());
+
+    final Class<?> returnType =
+        invoker.getReturnType(request.getOperation(), request.getParameterTypes());
+

Review Comment:
   request.getOperation() is called twice here can we call it once store in a variable and use that.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/invoker/ScmInvokerCodeGenerator.java:
##########
@@ -442,6 +448,73 @@ void printProxyMethod() {
     }
   }
 
+  void printReturnTypeMethod() {
+    println();
+    println("@Override");
+    printf("public Class<?> getReturnType(String methodName, Class<?>[] parameterTypes)");
+    try (UncheckedAutoCloseable ignored = printScope()) {
+      printReturnTypeSwitch();
+    }
+  }
+
+  void printReturnTypeSwitch() {
+    printf("switch (methodName)");
+    try (UncheckedAutoCloseable ignored = printScope(true, 0)) {
+      final List<Method> apiMethods = getReturnTypeMethods();
+
+      final List<String> methodNames = apiMethods.stream()
+          .map(Method::getName)
+          .distinct()
+          .sorted()
+          .collect(Collectors.toList());

Review Comment:
   it is already sorted by getReturnTypeMethods()



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3088225030


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/invoker/DeletedBlockLogStateManagerInvoker.java:
##########
@@ -115,7 +119,9 @@ public Object invokeLocal(String methodName, Object[] p) throws Exception {
       return null;

Review Comment:
   @Russole , For void.class, we probably should return `Message.EMPTY`.  Sorry that I missed it last time.
   
   > Further improvements to ScmInvokerCodeGenerator will be handled in a separate JIRA.
   
   I agree to change ScmInvokerCodeGenerator separately after we make sure that the code change is good.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #10079:
URL: https://github.com/apache/ozone/pull/10079#issuecomment-4267803756

   Thanks @Russole for the patch, @sreejasahithi, @szetszwo for the review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3088225030


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/invoker/DeletedBlockLogStateManagerInvoker.java:
##########
@@ -115,7 +119,9 @@ public Object invokeLocal(String methodName, Object[] p) throws Exception {
       return null;

Review Comment:
   @Russole , For void.class, we should return `Message.EMPTY`.  Sorry that I missed it last time.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #10079:
URL: https://github.com/apache/ozone/pull/10079


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3081342816


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMStateMachine.java:
##########
@@ -179,6 +187,10 @@ public CompletableFuture<Message> applyTransaction(
   }
 
   private Message process(final SCMRatisRequest request) throws Exception {
+    final ScmInvoker<?> invoker = invokers.get(request.getType());
+    if (invoker != null) {
+      return process(request, invoker);

Review Comment:
   Let's change ScmInvoker.invokeLocal(..) to return message.  Then, we don't need to new `process` method:
   ```java
     private Message process(final SCMRatisRequest request) throws Exception {
       final ScmInvoker<?> invoker = invokers.get(request.getType());
       if (invoker != null) {
         return invoker.invokeLocal(request.getOperation(), request.getArguments());
       }
       return process(request, handlers.get(request.getType()));
     }
   ```



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMStateMachine.java:
##########
@@ -117,6 +120,11 @@ public void registerHandler(RequestType type, Object handler) {
     handlers.put(type, handler);
   }
 
+  public void registerStateMachineInvoker(RequestType type,

Review Comment:
   Let's simply name it `registerInvoker`.
   
   BTW, this method is not called.  We need to update SCMRatisServerImpl:
   ```diff
   @@ -223,7 +224,11 @@ public SCMStateMachine getSCMStateMachine() {
      @Override
      public void registerStateMachineHandler(final RequestType handlerType,
                                              final Object handler) {
   -    stateMachine.registerHandler(handlerType, handler);
   +    if (handler instanceof ScmInvoker) {
   +      stateMachine.registerInvoker(handlerType, (ScmInvoker) handler);
   +    } else {
   +      stateMachine.registerHandler(handlerType, handler);
   +    }
      }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "sreejasahithi (via GitHub)" <gi...@apache.org>.
sreejasahithi commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3084172297


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMStateMachine.java:
##########
@@ -77,6 +78,7 @@ public class SCMStateMachine extends BaseStateMachine {
 
   private StorageContainerManager scm;
   private Map<RequestType, Object> handlers;
+  private Map<RequestType,  ScmInvoker<?>> invokers;

Review Comment:
   ```suggestion
     private Map<RequestType, ScmInvoker<?>> invokers;
   ```



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/invoker/ScmInvokerCodeGenerator.java:
##########
@@ -72,6 +72,12 @@ public final class ScmInvokerCodeGenerator {
       Object.class,
       new Class<?>[]{Exception.class});
 
+  static final DeclaredMethod GET_RETURN_TYPE = new DeclaredMethod("getReturnType",
+      new Class[]{String.class, Class[].class},
+      new String[]{"methodName", "parameterTypes"},
+      Class.class,
+      new Class<?>[]{});
+

Review Comment:
   This is not used anywhere.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/invoker/ScmInvokerCodeGenerator.java:
##########


Review Comment:
   Not related to your change but ,
   here it should be tmp.getAbsolutePath() instead of java.getAbsolutePath()



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3091599435


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMRatisServerImpl.java:
##########
@@ -223,6 +224,9 @@ public SCMStateMachine getSCMStateMachine() {
   @Override
   public void registerStateMachineHandler(final RequestType handlerType,
                                           final Object handler) {
+    if (handler instanceof ScmInvoker) {
+      stateMachine.registerInvoker(handlerType, (ScmInvoker) handler);
+    }
     stateMachine.registerHandler(handlerType, handler);

Review Comment:
   We should add `else`.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMRatisServerImpl.java:
##########
@@ -223,6 +224,9 @@ public SCMStateMachine getSCMStateMachine() {
   @Override
   public void registerStateMachineHandler(final RequestType handlerType,
                                           final Object handler) {
+    if (handler instanceof ScmInvoker) {
+      stateMachine.registerInvoker(handlerType, (ScmInvoker) handler);

Review Comment:
   You are right -- we need to pass invoker but not getImpl().
   ```java
   registerStateMachineHandler(invoker.getType(), invoker);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14974. Change SCMStateMachine to use ScmInvoker [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #10079:
URL: https://github.com/apache/ozone/pull/10079#discussion_r3091514424


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMRatisServerImpl.java:
##########
@@ -223,6 +224,9 @@ public SCMStateMachine getSCMStateMachine() {
   @Override
   public void registerStateMachineHandler(final RequestType handlerType,
                                           final Object handler) {
+    if (handler instanceof ScmInvoker) {
+      stateMachine.registerInvoker(handlerType, (ScmInvoker) handler);

Review Comment:
   I don't think this method is called with `ScmInvoker`, rather `invoker.getImpl()`.
   
   https://github.com/apache/ozone/blob/874dad783b7dbe319edde54da4e519901a8f2f3a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMRatisServer.java#L75-L76
   
   ```
   [main] INFO ha.SCMRatisServerImpl: registerStateMachineHandler(FINALIZE, org.apache.hadoop.hdds.scm.server.upgrade.FinalizationStateManagerImpl@273a5a8a): invoker=false
   [main] INFO ha.SCMRatisServerImpl: registerStateMachineHandler(SEQUENCE_ID, org.apache.hadoop.hdds.scm.ha.SequenceIdGenerator$StateManagerImpl@76544c0a): invoker=false
   [main] INFO ha.SCMRatisServerImpl: registerStateMachineHandler(PIPELINE, org.apache.hadoop.hdds.scm.pipeline.PipelineStateManagerImpl@57a6a933): invoker=false
   [main] INFO ha.SCMRatisServerImpl: registerStateMachineHandler(CONTAINER, org.apache.hadoop.hdds.scm.container.ContainerStateManagerImpl@31f295b6): invoker=false
   [main] INFO ha.SCMRatisServerImpl: registerStateMachineHandler(BLOCK, org.apache.hadoop.hdds.scm.block.DeletedBlockLogStateManagerImpl@423a0e1d): invoker=false
   [main] INFO ha.SCMRatisServerImpl: registerStateMachineHandler(STATEFUL_SERVICE_CONFIG, org.apache.hadoop.hdds.scm.ha.StatefulServiceStateManagerImpl@1c240cf2): invoker=false
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org