You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by ja...@apache.org on 2022/07/25 21:59:56 UTC
[pinot] branch master updated: Do not use PartitionUpsertMetadataManager.replaceSegment (#9101)
This is an automated email from the ASF dual-hosted git repository.
jackie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 0c1037ed90 Do not use PartitionUpsertMetadataManager.replaceSegment (#9101)
0c1037ed90 is described below
commit 0c1037ed90d75bb7cd95315cd6a6bdd00f34a6c2
Author: Xiaotian (Jackie) Jiang <17...@users.noreply.github.com>
AuthorDate: Mon Jul 25 14:59:51 2022 -0700
Do not use PartitionUpsertMetadataManager.replaceSegment (#9101)
---
.../pinot/core/data/manager/realtime/RealtimeTableDataManager.java | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java b/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
index 0a09ff4087..654133af11 100644
--- a/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
+++ b/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
@@ -430,7 +430,12 @@ public class RealtimeTableDataManager extends BaseTableDataManager {
_logger.info("Added new immutable segment: {} to upsert-enabled table: {}", segmentName, _tableNameWithType);
} else {
IndexSegment oldSegment = oldSegmentManager.getSegment();
- partitionUpsertMetadataManager.replaceSegment(immutableSegment, oldSegment);
+ partitionUpsertMetadataManager.addSegment(immutableSegment);
+ // TODO: Fix the following issue about replacing segment in upsert metadata
+ // - We cannot directly invalidate the docs in the replaced segment because query might still running against it
+ // - We should track the valid docs in the replaced segment separately. Currently the docs won't be invalidate
+ // in the replaced segment due to the reason above, and will cause wrong logs/metrics emitted.
+ // partitionUpsertMetadataManager.replaceSegment(immutableSegment, oldSegment);
_logger.info("Replaced {} segment: {} of upsert-enabled table: {}",
oldSegment instanceof ImmutableSegment ? "immutable" : "mutable", segmentName, _tableNameWithType);
releaseSegment(oldSegmentManager);
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org