Re: [I] store partitioning in dynamic filters and remove CASE hash(expr) [datafusion]

["stuhood (via GitHub)" <gi...@apache.org>]

stuhood commented on issue #23817:
URL: https://github.com/apache/datafusion/issues/23817#issuecomment-5098537573

   > I'm not 100% sure if that's a problem since dynamic filters already store partitioning, baked into the `CASE` statement. Storing it separately should behave similarly.
   
   The difference (I think) is that a `CASE` expression is a "total" function: it correctly handles all inputs that have been given to it, even if it hasn't been repartitioned. Whereas each expression inside a partition-aware dynamic filter would be "partial": if you don't correctly choose which expression to evaluate, you produce the wrong result.
   
   But as mentioned, I think that asserts could be sufficient.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org