You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "deepthi912 (via GitHub)" <gi...@apache.org> on 2026/02/13 04:48:26 UTC
Re: [PR] Add consensus check before selecting a segment for compaction or deletion [pinot]
deepthi912 commented on code in PR #17352:
URL: https://github.com/apache/pinot/pull/17352#discussion_r2802255970
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/MinionTaskUtils.java:
##########
@@ -334,4 +335,47 @@ public static ValidDocIdsType getValidDocIdsType(UpsertConfig upsertConfig, Map<
}
return validDocIdsType;
}
+
+ /**
+ * Checks if all replicas have consensus on validDoc counts for a segment.
+ * SAFETY LOGIC:
+ * 1. Only proceed with operations when ALL replicas agree on totalValidDocs count
+ * 2. Skip operations if ANY server hosting the segment is not in READY state
+ * 3. Include all replicas (even those with CRC mismatches) in consensus for safety
+ *
+ * @param segmentName the name of the segment being checked
+ * @param replicaMetadataList list of metadata from all replicas of the segment
+ * @return true if all replicas have consensus on validDoc counts, false otherwise
+ */
+ public static boolean hasValidDocConsensus(String segmentName,
Review Comment:
Good point to agree on the same validDocs, but sometimes servers might be at different points of snapshots and not having same count is highly possible, In such cases if we could pick the ones with higher validDocs might make more sense no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org