You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xianyang Liu (Jira)" <ji...@apache.org> on 2023/08/22 14:59:00 UTC
[jira] [Created] (SPARK-44913) DS V2 supports push down V2 UDF that has magic method
Xianyang Liu created SPARK-44913:
------------------------------------
Summary: DS V2 supports push down V2 UDF that has magic method
Key: SPARK-44913
URL: https://issues.apache.org/jira/browse/SPARK-44913
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.4.1
Reporter: Xianyang Liu
Right now we only support pushing down the V2 UDF that has not a magic method. Because the V2 UDF will be analyzed into the `ApplyFunctionExpression` which could be translated and pushed down. However, a V2 UDF that has the magic method will be analyzed into `StaticInvoke` or `Invoke` that can not be translated into V2 expression and then can not be pushed down to the data source. The magic method is suggested. So this PR adds the support of pushing down the V2 UDF that has a magic method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org