[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10290: Add a tight retry block to update ZK during endSegmentReplacement

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on code in PR #10290:
URL: https://github.com/apache/pinot/pull/10290#discussion_r1108916745


##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -3473,23 +3458,41 @@ public void endReplaceSegments(String tableNameWithType, String segmentLineageEn
           return false;
         }
 
-        // Update lineage entry
-        LineageEntry newLineageEntry =
-            new LineageEntry(lineageEntry.getSegmentsFrom(), segmentsTo, LineageEntryState.COMPLETED,
-                System.currentTimeMillis());
-        segmentLineage.updateLineageEntry(segmentLineageEntryId, newLineageEntry);
+        // Start the tighter loop for ZK Update to avoid too frequent write failure.
+        DEFAULT_RETRY_POLICY.attempt(() -> {
+          // Fetch the segment lineage again
+          ZNRecord segmentLineageToUpdateZNRecord =
+              SegmentLineageAccessHelper.getSegmentLineageZNRecord(_propertyStore, tableNameWithType);
+          int expectedVersion = segmentLineageToUpdateZNRecord.getVersion();
+          SegmentLineage segmentLineageToUpdate = SegmentLineage.fromZNRecord(segmentLineageToUpdateZNRecord);
+          LineageEntry lineageEntryToUpdate = segmentLineageToUpdate.getLineageEntry(segmentLineageEntryId);
+
+          // Check if we need to proceed with updating the lineage entry based on its current state.
+          if (!shouldUpdateLineageEntry(tableNameWithType, segmentLineageEntryId, lineageEntry)) {

Review Comment:
   We don't want to do this check again. We should check if the new fetched lineage entry is the same as the previous one, and then set the new lineage entry and post it back



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org