[GitHub] [pinot] KKcorps commented on a diff in pull request #10826: Upsert snapshot should be taken before new a consuming segment created

["KKcorps (via GitHub)" <gi...@apache.org>]

KKcorps commented on code in PR #10826:
URL: https://github.com/apache/pinot/pull/10826#discussion_r1217731384


##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java:
##########
@@ -662,6 +659,46 @@ public void run() {
       long catchUpTimeMillis = 0L;
       _startTimeMs = now();
       try {
+        if (!_isReadyToConsumeData.getAsBoolean()) {
+          do {
+            //noinspection BusyWait
+            Thread.sleep(RealtimeTableDataManager.READY_TO_CONSUME_DATA_CHECK_INTERVAL_MS);
+          } while (!_shouldStop && !endCriteriaReached() && !_isReadyToConsumeData.getAsBoolean());
+        }
+
+        if (_tableConfig.getUpsertConfig() != null && _tableConfig.getUpsertConfig().isEnableSnapshot()) {
+          // Persist snapshot for sealed segments. We need to guarantee the previous segment is already replaced with
+          // the immutable segment, so the snapshot might not be persisted for the previous consuming segment.
+          List<SegmentDataManager> allSegments = _realtimeTableDataManager.acquireAllSegments();
+          List<SegmentDataManager> allSegmentsForPartition = _realtimeTableDataManager.acquireAllSegments();

Review Comment:
   What's the difference between this line and the previous one except the variable name?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org