Re: [PR] [core] Optimize BTree negative predicate complement [paimon]

["zhoulii (via GitHub)" <gi...@apache.org>]

zhoulii commented on code in PR #8945:
URL: https://github.com/apache/paimon/pull/8945#discussion_r3701051812


##########
paimon-common/src/main/java/org/apache/paimon/globalindex/btree/LazyFilteredBTreeReader.java:
##########
@@ -56,6 +56,11 @@ public LazyFilteredBTreeReader(
         this.keySerializer = keySerializer;
     }
 
+    @Override
+    public boolean supportsRangeComplement() {

Review Comment:
   Do you mean overriding visitIsNotNull, visitNotEqual, and visitNotIn directly in LazyFilteredBTreeReader?
   
   The challenge is that LazyFilteredBTreeReader only has the index files and local row IDs, but it does not know the complete row-id range of the shard. A correct complement requires this full range, which is currently only available in OffsetGlobalIndexReader as [offset, to].
   
   Implementing the complement directly in LazyFilteredBTreeReader would either require reading all BTree files to reconstruct the full row-id set, which defeats this optimization, or passing the row range through GlobalIndexer#createReader.
   
   That is why the current implementation computes the complement in OffsetGlobalIndexReader and lets LazyFilteredBTreeReader opt in through supportsRangeComplement().



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org