You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/12/19 00:15:00 UTC

[jira] [Updated] (SPARK-50618) Refactor Data[Frame|Stream]Reader to leverage the analyzer

     [ https://issues.apache.org/jira/browse/SPARK-50618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-50618:
-----------------------------------
    Labels: pull-request-available  (was: )

> Refactor Data[Frame|Stream]Reader to leverage the analyzer
> ----------------------------------------------------------
>
>                 Key: SPARK-50618
>                 URL: https://issues.apache.org/jira/browse/SPARK-50618
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 4.0.0
>            Reporter: Burak Yavuz
>            Assignee: Burak Yavuz
>            Priority: Major
>              Labels: pull-request-available
>
> The DataFrameReader and DataStreamReader typically creates analyzed relations as part of their respective `.load()` methods.
> This creates inconsistencies for what rules get applied to the query plan as part of Catalyst depending on your API of choice, such as SQL vs Python or SQL vs Scala.
> The goal of this Jira is to refactor the logic in DataFrameReader and DataStreamReader classes to create unresolved plans that get analyzed as part of Catalyst.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org