[jira] [Resolved] (SPARK-58261) Support for SCD Type 2 in SQL syntax

["Szehon Ho (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-58261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Szehon Ho resolved SPARK-58261.
-------------------------------
    Fix Version/s: 4.3.0
       Resolution: Fixed

Issue resolved by pull request 57434
[https://github.com/apache/spark/pull/57434]

> Support for SCD Type 2 in SQL syntax
> ------------------------------------
>
>                 Key: SPARK-58261
>                 URL: https://issues.apache.org/jira/browse/SPARK-58261
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Declarative Pipelines, SQL
>    Affects Versions: 4.3.0
>            Reporter: Andreas Neumann
>            Assignee: Andreas Neumann
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.3.0
>
>
> To support SCD Type 2, SQL must allow:
> - STORED AS SCD TYPE (1|2)
> - TRACK HISTORY ON (<column names> | * EXCEPT <column names>)
> STORED AS SCD TYPE is optional and defaults to 1. 
> TRACK HISTORY is only allowed only for SCD2. 
> Also, TRACK HISTORY must be the last clause in the AutoCdc command.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org