You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Daniel (Jira)" <ji...@apache.org> on 2022/05/03 23:07:00 UTC

[jira] [Commented] (SPARK-38869) Respect Table capability `ACCEPT_ANY_SCHEMA` in default column resolution

    [ https://issues.apache.org/jira/browse/SPARK-38869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531444#comment-17531444 ] 

Daniel commented on SPARK-38869:
--------------------------------

Thanks for pointing this out. We can skip adding new columns with default values if the inserted relation has fewer columns than the target table. If any of the inserted columns are explicit DEFAULT references that would have otherwise failed analysis, we can replace them with corresponding values as with other tables (either the provided DEFAULT values from the target schema, or NULL otherwise).

> Respect Table capability `ACCEPT_ANY_SCHEMA` in default column resolution
> -------------------------------------------------------------------------
>
>                 Key: SPARK-38869
>                 URL: https://issues.apache.org/jira/browse/SPARK-38869
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.4.0
>            Reporter: Gengliang Wang
>            Assignee: Daniel
>            Priority: Major
>
> If a V2 table has the capability of [ACCEPT_ANY_SCHEMA|https://github.com/apache/spark/blob/master/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/TableCapability.java#L94], we should skip adding default column values to the insert schema.
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org