[GitHub] [spark] Kimahriman commented on pull request #34558: [SPARK-37019][SQL] Add codegen support to array higher-order functions

[GitBox <gi...@apache.org>]

Kimahriman commented on pull request #34558:
URL: https://github.com/apache/spark/pull/34558#issuecomment-1067900687


   It's hard to say because when I tested this out on my production jobs (actually still actively using it), I had several other changes too. I'm not sure if there are any benchmarks involving HOFs? Though it's highly dependent on what the lambda function is, and honestly that's one of the main benefits, the lambda functions themselves can be codegen'd instead of eval'd. 
   
   I also have a larger goal to support subexpression elimination inside lambda functions, because that's where I've found our biggest problem is. https://github.com/apache/spark/pull/34727 is also part of that goal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org