[jira] [Updated] (SPARK-29966) avoid load table twice

["Wenchen Fan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-29966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan updated SPARK-29966:
--------------------------------
    Summary: avoid load table twice  (was: Add version method in TableCatalog to avoid load table twice)

> avoid load table twice
> ----------------------
>
>                 Key: SPARK-29966
>                 URL: https://issues.apache.org/jira/browse/SPARK-29966
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: ulysses you
>            Assignee: Terry Kim
>            Priority: Minor
>             Fix For: 3.0.0
>
>
> Now resolve logic plan will load table twice which are in ResolveTables and ResolveRelations. The ResolveRelations is old code path, and ResolveTables is v2 code path, and the reason why load table twice is that ResolveTables will load table and rollback v1 table to ResolveRelations code path.
> The same scene also exists in ResolveSessionCatalog.
> It affect that execute command will cost double time than spark 2.4.
> Here is the idea that add a table version method in TableCatalog, and rules should always get table version firstly without load table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org