[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7971: Fix to query inconsistencies under heavy upsert load (resolves #7958)

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on a change in pull request #7971:
URL: https://github.com/apache/pinot/pull/7971#discussion_r784333878



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/plan/FilterPlanNode.java
##########
@@ -52,19 +52,30 @@
 import org.apache.pinot.segment.spi.index.reader.JsonIndexReader;
 import org.apache.pinot.segment.spi.index.reader.NullValueVectorReader;
 import org.apache.pinot.spi.exception.BadQueryRequestException;
+import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
 
 public class FilterPlanNode implements PlanNode {
   private final IndexSegment _indexSegment;
   private final QueryContext _queryContext;
   private final int _numDocs;
+  private final MutableRoaringBitmap _validDocIds;
 
   // Cache the predicate evaluators
   private final Map<Predicate, PredicateEvaluator> _predicateEvaluatorMap = new HashMap<>();
 
   public FilterPlanNode(IndexSegment indexSegment, QueryContext queryContext) {
     _indexSegment = indexSegment;
     _queryContext = queryContext;
+
+    // Snapshot validDocIds *before* _numDocs since otherwise newest upserts could get lost

Review comment:
       Taking snapshot when creating the plan nodes can potentially cause performance issue since it is done single-threaded




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org