[GitHub] [pinot] richardstartin commented on a change in pull request #7971: Fix to query inconsistencies under heavy upsert load (resolves #7958)

[GitBox <gi...@apache.org>]

richardstartin commented on a change in pull request #7971:
URL: https://github.com/apache/pinot/pull/7971#discussion_r785424027



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/plan/FilterPlanNode.java
##########
@@ -52,19 +52,30 @@
 import org.apache.pinot.segment.spi.index.reader.JsonIndexReader;
 import org.apache.pinot.segment.spi.index.reader.NullValueVectorReader;
 import org.apache.pinot.spi.exception.BadQueryRequestException;
+import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
 
 public class FilterPlanNode implements PlanNode {
   private final IndexSegment _indexSegment;
   private final QueryContext _queryContext;
   private final int _numDocs;
+  private final MutableRoaringBitmap _validDocIds;
 
   // Cache the predicate evaluators
   private final Map<Predicate, PredicateEvaluator> _predicateEvaluatorMap = new HashMap<>();
 
   public FilterPlanNode(IndexSegment indexSegment, QueryContext queryContext) {
     _indexSegment = indexSegment;
     _queryContext = queryContext;
+
+    // Snapshot validDocIds *before* _numDocs since otherwise newest upserts could get lost

Review comment:
       In any case I know how to make snapshots faster in the library given that the bitmap is only ever appended to, so we could do that work if it ever showed up in profiles.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org