You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Holden Karau (Jira)" <ji...@apache.org> on 2023/07/27 23:53:00 UTC

[jira] [Created] (SPARK-44578) Support pushing down UDFs in DSv2

Holden Karau created SPARK-44578:
------------------------------------

             Summary: Support pushing down UDFs in DSv2
                 Key: SPARK-44578
                 URL: https://issues.apache.org/jira/browse/SPARK-44578
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.5.0, 4.0.0
            Reporter: Holden Karau
            Assignee: Holden Karau


We should consider trying to add support for pushing down UDFS to the storage engine. While most of the time this might not make sense, some storage engines expose their own UDFS like bucketing or day transformers, which we would ideally push down to them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org