Re: [PR] [SPARK-47547] BloomFilter fpp degradation [spark]

["ishnagy (via GitHub)" <gi...@apache.org>]

ishnagy commented on PR #50933:
URL: https://github.com/apache/spark/pull/50933#issuecomment-3026954868

   > Bloom filter functions (BloomFilterAggregate / BloomFilterMightContain) seem like internal ones so I wouldn't expect serialized filters to be shared between different application runs, but it seems some native accelerators [reimplemented](https://github.com/apache/datafusion-comet/blob/main/native/core/src/execution/util/spark_bloom_filter.rs#L63) Spark's logic (including its flaws) so we should probably use a new [version](https://github.com/apache/spark/blob/master/common/sketch/src/main/java/org/apache/spark/util/sketch/BloomFilterImpl.java#L241) with this improvement.
   
   ok, I'll try to provide an update by EOW.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org