[jira] [Resolved] (SPARK-33138) unify temp view and permanent view behaviors

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-33138.
----------------------------------
    Fix Version/s: 3.1.1
       Resolution: Done

I am resolving this umbrella JIRA targeting 3.1.1. Please file another umbrella and link this ticket if there are some more issues to fix.

> unify temp view and permanent view behaviors
> --------------------------------------------
>
>                 Key: SPARK-33138
>                 URL: https://issues.apache.org/jira/browse/SPARK-33138
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Leanken.Lin
>            Priority: Major
>             Fix For: 3.1.1
>
>
> Currently, temp view store mapping of temp view name and its logicalPlan, and permanent view store in HMS stores its origin SQL text.
> So for permanent view, when try to refer the permanent view, its SQL text will be parse-analyze-optimize-plan again with current SQLConf and SparkSession context, so it might keep changing when the SQLConf and context is different each time.
> In order the unify the behaviors of temp view and permanent view, proposed that we keep its origin SQLText for both temp and permanent view, and also keep record of the SQLConf when the view was created. Each time we try to refer the view, we using the Snapshot SQLConf to parse-analyze-optimize-plan the SQLText, in this way, we can make sure the output of the created view to be stable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org