You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Xinrong Meng (Jira)" <ji...@apache.org> on 2025/01/07 01:37:00 UTC

[jira] [Updated] (SPARK-50391) Support DataFrame conversion to table argument

     [ https://issues.apache.org/jira/browse/SPARK-50391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xinrong Meng updated SPARK-50391:
---------------------------------
    Summary: Support DataFrame conversion to table argument  (was: Support DataFrame conversion to table argument in Spark Classic)

> Support DataFrame conversion to table argument
> ----------------------------------------------
>
>                 Key: SPARK-50391
>                 URL: https://issues.apache.org/jira/browse/SPARK-50391
>             Project: Spark
>          Issue Type: Umbrella
>          Components: Connect, PySpark, SQL
>    Affects Versions: 4.0.0
>            Reporter: Xinrong Meng
>            Priority: Major
>
> Table-Valued Functions (TVFs) and User-Defined Table Functions (UDTFs) are widely used in Spark workflows. These functions often require a table argument, which Spark internally represents as a Catalyst expression. While Spark SQL supports constructs like TABLE(<query>) for this purpose, **there is no direct API in PySpark or Scala to convert a DataFrame into a table argument**. So we propose to support DataFrame conversion to table arguments (in Spark Classic first), and enable UDTFs to accept table arguments in both PySpark and Scala..



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org