You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Anton Okolnychyi (Jira)" <ji...@apache.org> on 2021/06/22 01:35:00 UTC
[jira] [Updated] (SPARK-35801) SPIP: Support MERGE in Data Source
V2
[ https://issues.apache.org/jira/browse/SPARK-35801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anton Okolnychyi updated SPARK-35801:
-------------------------------------
Description:
[MERGE INTO|https://en.wikipedia.org/wiki/Merge_(SQL)] is well suited to large-scale workloads because it can express operations to insert, update, or delete multiple rows in a single SQL command. Many updates can be expressed as MERGE INTO queries that would otherwise require much more SQL. Common patterns for updating partitions are to read, union, and overwrite or read, diff, and append. Using MERGE INTO, these operations are easier to express and can be more efficient to run.
Hive supports [MERGE INTO|https://blog.cloudera.com/update-hive-tables-easy-way/] and Spark should implement similar support.
SPIP: https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60
was:
[MERGE INTO|https://en.wikipedia.org/wiki/Merge_(SQL)] is well suited to large-scale workloads because it can express operations to insert, update, or delete multiple rows in a single SQL command. Many updates can be expressed as MERGE INTO queries that would otherwise require much more SQL. Common patterns for updating partitions are to read, union, and overwrite or read, diff, and append. Using MERGE INTO, these operations are easier to express and can be more efficient to run.
Hive supports [MERGE INTO|https://blog.cloudera.com/update-hive-tables-easy-way/] and Spark should implement similar support.
> SPIP: Support MERGE in Data Source V2
> -------------------------------------
>
> Key: SPARK-35801
> URL: https://issues.apache.org/jira/browse/SPARK-35801
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Anton Okolnychyi
> Priority: Major
>
> [MERGE INTO|https://en.wikipedia.org/wiki/Merge_(SQL)] is well suited to large-scale workloads because it can express operations to insert, update, or delete multiple rows in a single SQL command. Many updates can be expressed as MERGE INTO queries that would otherwise require much more SQL. Common patterns for updating partitions are to read, union, and overwrite or read, diff, and append. Using MERGE INTO, these operations are easier to express and can be more efficient to run.
> Hive supports [MERGE INTO|https://blog.cloudera.com/update-hive-tables-easy-way/] and Spark should implement similar support.
> SPIP: https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org