[GitHub] [pinot] Jackie-Jiang commented on issue #7913: Increase DEFAULT_VALUE_PRUNER_IN_PREDICATE_THRESHOLD or derive the value with some simple policy

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on issue #7913:
URL: https://github.com/apache/pinot/issues/7913#issuecomment-996165566


   The problem here is that:
   If segment cannot be pruned, we are paying the overhead of parsing all the values in the IN clause and apply it to bloom filter (assume comparing min/max value is very cheap)
   If segment can be pruned, even if we skip the pruning because there are too many values in the IN clause, the segment will still be quickly skipped after looking up the values in the dictionary and find there is no match. So I don't think the assumption of full table scan stands here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org