[GitHub] [spark] beliefer commented on pull request #36593: [SPARK-39139][SQL] DS V2 push-down framework supports DS V2 UDF

[GitBox <gi...@apache.org>]

beliefer commented on PR #36593:
URL: https://github.com/apache/spark/pull/36593#issuecomment-1143143171

   > glad to see interest/progress in cross platform SQL/UDFs pushdown. Have you considered doing this leveraging frameworks such as Transport [[1](https://github.com/linkedin/transport), [2](https://engineering.linkedin.com/blog/2018/11/using-translatable-portable-UDFs)] for UDFs and Coral [[1](https://github.com/linkedin/coral), [2](https://engineering.linkedin.com/blog/2020/coral)] for SQL?
   
   Spark DS V2 has the special UDF API. Users could implement UDF with the API and DS V2 push-down framework could supports them. If we use Transport and Coral, it seems introduce more components.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org