You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "sarvekshayr (via GitHub)" <gi...@apache.org> on 2025/05/08 16:15:57 UTC

[PR] HDDS-12970. Use DatanodeID in Pipeline [ozone]

sarvekshayr opened a new pull request, #8414:
URL: https://github.com/apache/ozone/pull/8414

   ## What changes were proposed in this pull request?
   Use `DatanodeID` instead of `UUID` for `leaderId` and `suggestedLeaderId` in `Pipeline`.
   
   ## What is the link to the Apache JIRA
   [HDDS-12970](https://issues.apache.org/jira/browse/HDDS-12970)
   
   ## How was this patch tested?
   Updated existing tests.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12970. Use DatanodeID in Pipeline [ozone]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo merged PR #8414:
URL: https://github.com/apache/ozone/pull/8414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12970. Use DatanodeID in Pipeline [ozone]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #8414:
URL: https://github.com/apache/ozone/pull/8414#discussion_r2080131929


##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java:
##########
@@ -451,20 +444,20 @@ public static Builder toBuilder(HddsProtos.Pipeline pipeline) {
       nodes.put(DatanodeDetails.getFromProtoBuf(member), repIndex);
       index++;
     }
-    UUID leaderId = null;
+    DatanodeID leaderId = null;
     if (pipeline.hasLeaderID128()) {
       HddsProtos.UUID uuid = pipeline.getLeaderID128();
-      leaderId = new UUID(uuid.getMostSigBits(), uuid.getLeastSigBits());
+      leaderId = DatanodeID.of(new UUID(uuid.getMostSigBits(), uuid.getLeastSigBits()));

Review Comment:
   Add a new `DatanodeID.of(..)` method.
   ```java
   +++ b/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocol/DatanodeID.java
   @@ -94,6 +94,10 @@ public static DatanodeID of(final UUID id) {
        return CACHE.computeIfAbsent(id, DatanodeID::new);
      }
    
   +  public static DatanodeID of(final HddsProtos.UUID uuid) {
   +    return of(new UUID(uuid.getMostSigBits(), uuid.getLeastSigBits()));
   +  }
   +
      /**
       * Returns a random DatanodeID.
       */
   ```



##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java:
##########
@@ -451,20 +444,20 @@ public static Builder toBuilder(HddsProtos.Pipeline pipeline) {
       nodes.put(DatanodeDetails.getFromProtoBuf(member), repIndex);
       index++;
     }
-    UUID leaderId = null;
+    DatanodeID leaderId = null;
     if (pipeline.hasLeaderID128()) {
       HddsProtos.UUID uuid = pipeline.getLeaderID128();
-      leaderId = new UUID(uuid.getMostSigBits(), uuid.getLeastSigBits());
+      leaderId = DatanodeID.of(new UUID(uuid.getMostSigBits(), uuid.getLeastSigBits()));
     } else if (pipeline.hasLeaderID() &&
         StringUtils.isNotEmpty(pipeline.getLeaderID())) {
-      leaderId = UUID.fromString(pipeline.getLeaderID());
+      leaderId = DatanodeID.of(UUID.fromString(pipeline.getLeaderID()));

Review Comment:
   Use `DatanodeID.fromUuidString(..)`.



##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/PipelineEndpoint.java:
##########
@@ -30,6 +30,7 @@
 import javax.ws.rs.core.MediaType;
 import javax.ws.rs.core.Response;
 import org.apache.hadoop.hdds.protocol.DatanodeDetails;
+import org.apache.hadoop.hdds.protocol.DatanodeID;

Review Comment:
   Remove UUID import:
   ```diff
   +++ b/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/PipelineEndpoint.java
   @@ -22,7 +22,6 @@
    import java.util.ArrayList;
    import java.util.List;
    import java.util.TreeMap;
   -import java.util.UUID;
    import javax.inject.Inject;
    import javax.ws.rs.GET;
    import javax.ws.rs.Path;
   @@ -32,6 +31,7 @@
    import org.apache.hadoop.hdds.protocol.DatanodeDetails;
    import org.apache.hadoop.hdds.protocol.DatanodeID;
    import org.apache.hadoop.hdds.scm.pipeline.Pipeline;
   +import org.apache.hadoop.hdds.scm.pipeline.PipelineID;
    import org.apache.hadoop.hdds.scm.server.OzoneStorageContainerManager;
    import org.apache.hadoop.ozone.recon.MetricsServiceProviderFactory;
    import org.apache.hadoop.ozone.recon.api.types.PipelineMetadata;
   @@ -74,7 +74,7 @@ public Response getPipelines() {
        List<Pipeline> pipelines = pipelineManager.getPipelines();
    
        pipelines.forEach(pipeline -> {
   -      UUID pipelineId = pipeline.getId().getId();
   +      final PipelineID pipelineId = pipeline.getId();
          List<DatanodeDetails> datanodes = new ArrayList<>();
          PipelineMetadata.Builder builder = PipelineMetadata.newBuilder();
          pipeline.getNodes().forEach(node -> datanodes.add(node));
   @@ -99,7 +99,7 @@ public Response getPipelines() {
          }
    
          PipelineMetadata.Builder pipelineBuilder =
   -          builder.setPipelineId(pipelineId)
   +          builder.setPipelineId(pipelineId.getId())
                  .setDatanodes(datanodes)
                  .setDuration(duration)
                  .setStatus(pipeline.getPipelineState())
   @@ -111,7 +111,7 @@ public Response getPipelines() {
            // Extract last part of pipelineId to get its group Id.
            // ex. group id of 48981bf7-8bea-4fbd-9857-79df51ee872d
            // is group-79DF51EE872D
   -        String[] splits = pipelineId.toString().split("-");
   +        String[] splits = pipelineId.getId().toString().split("-");
            String groupId = "group-" + splits[splits.length - 1].toUpperCase();
            Long leaderElectionCount = getElectionCountMetricValue(groupId);
            pipelineBuilder.setLeaderElections(leaderElectionCount);
   ```



##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java:
##########
@@ -451,20 +444,20 @@ public static Builder toBuilder(HddsProtos.Pipeline pipeline) {
       nodes.put(DatanodeDetails.getFromProtoBuf(member), repIndex);
       index++;
     }
-    UUID leaderId = null;
+    DatanodeID leaderId = null;

Review Comment:
   In the current code, there is a bug that `leaderDatanodeID` (and `suggestedLeaderDatanodeID`) are not checked here.  We should fix it.  
   
   Cc @nandakumar131 
   
   ```diff
   +++ b/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java
   @@ -33,7 +33,6 @@
    import java.util.Objects;
    import java.util.Optional;
    import java.util.Set;
   -import java.util.UUID;
    import java.util.function.Function;
    import java.util.stream.Collectors;
    import org.apache.commons.lang3.StringUtils;
   @@ -445,19 +444,22 @@ public static Builder toBuilder(HddsProtos.Pipeline pipeline) {
          index++;
        }
        DatanodeID leaderId = null;
   -    if (pipeline.hasLeaderID128()) {
   +    if (pipeline.hasLeaderDatanodeID()) {
   +      leaderId = DatanodeID.of(pipeline.getLeaderDatanodeID().getUuid());
   +    } else if (pipeline.hasLeaderID128()) {
          HddsProtos.UUID uuid = pipeline.getLeaderID128();
   -      leaderId = DatanodeID.of(new UUID(uuid.getMostSigBits(), uuid.getLeastSigBits()));
   +      leaderId = DatanodeID.of(uuid);
        } else if (pipeline.hasLeaderID() &&
            StringUtils.isNotEmpty(pipeline.getLeaderID())) {
   -      leaderId = DatanodeID.of(UUID.fromString(pipeline.getLeaderID()));
   +      leaderId = DatanodeID.fromUuidString(pipeline.getLeaderID());
        }
    
        DatanodeID suggestedLeaderId = null;
   -    if (pipeline.hasSuggestedLeaderID()) {
   +    if (pipeline.hasSuggestedLeaderDatanodeID()) {
   +      suggestedLeaderId = DatanodeID.of(pipeline.getSuggestedLeaderDatanodeID().getUuid());
   +    } else if (pipeline.hasSuggestedLeaderID()) {
          HddsProtos.UUID uuid = pipeline.getSuggestedLeaderID();
   -      suggestedLeaderId =
   -          DatanodeID.of(new UUID(uuid.getMostSigBits(), uuid.getLeastSigBits()));
   +      suggestedLeaderId = DatanodeID.of(uuid);
        }
    
        final ReplicationConfig config = ReplicationConfig
   ```



##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/PipelineEndpoint.java:
##########
@@ -154,11 +155,11 @@ private Long getElectionCountMetricValue(String groupId) {
   }
 
   private Long getLastLeaderElectionElapsedTimeMetricValue(String groupId,
-                                                           UUID uuid) {
+                                                           DatanodeID dnId) {
     String metricsQuery = String.format(
             "query=ratis_leader_election_lastLeaderElectionElapsedTime{group=" +
                     "\"%s\",exported_instance=\"%s\"}", groupId,
-            uuid.toString());
+            dnId.toString());

Review Comment:
   Remove `toString()`.
   ```diff
   @@ -158,8 +158,7 @@ private Long getLastLeaderElectionElapsedTimeMetricValue(String groupId,
                                                               DatanodeID dnId) {
        String metricsQuery = String.format(
                "query=ratis_leader_election_lastLeaderElectionElapsedTime{group=" +
   -                    "\"%s\",exported_instance=\"%s\"}", groupId,
   -            dnId.toString());
   +                    "\"%s\",exported_instance=\"%s\"}", groupId, dnId);
        try {
          List<Metric> metrics = metricsServiceProvider.getMetricsInstant(
                  metricsQuery);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org