[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6483: Guard against multiple consuming segments for same partition

[GitBox <gi...@apache.org>]

jackjlli commented on a change in pull request #6483:
URL: https://github.com/apache/incubator-pinot/pull/6483#discussion_r564006471



##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
##########
@@ -793,6 +793,27 @@ void updateInstanceStatesForNewConsumingSegment(Map<String, Map<String, String>>
       LOGGER.info("Updating segment: {} to ONLINE state", committingSegmentName);
     }
 
+    // There used to be a race condition in pinot (caused by heavy GC on the controller during segment commit)
+    // that ended up creating multiple consuming segments for the same stream partition, named somewhat like
+    // tableName__1__25__20210920T190005Z and tableName__1__25__20210920T190007Z. It was fixed by checking the
+    // Zookeeper Stat object before updating the segment metadata.
+    // These conditions can happen again due to manual operations considered as fixes in Issues #5559 and #5263
+    // The following check prevents the table from going into such a state (but does not prevent the root cause
+    // of attempting such a zk update).
+    if (instanceStatesMap != null) {

Review comment:
       `instanceStateMap` could be null if the table is newly created, i.e. this method gets invoked by `setUpNewTable` method.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org