You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/07/02 03:27:00 UTC

[jira] [Commented] (IMPALA-11377) Handle concurrent Iceberg INSERT OVERWRITEs

    [ https://issues.apache.org/jira/browse/IMPALA-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561666#comment-17561666 ] 

ASF subversion and git services commented on IMPALA-11377:
----------------------------------------------------------

Commit ffd226cfeefcf8be15571f279fff90a860beeeac in impala's branch refs/heads/master from Tamas Mate
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ffd226cfe ]

IMPALA-11377: Handle concurrent Iceberg INSERT OVERWRITEs

Iceberg now supports validations in the ReplacePartitions API, an
initial snapshot id can be configured for the operation and Iceberg
will validate at commit if there were any conflicting change while the
ReplacePartitions was running.

This commit utilizes this new functionality, the Coordinator sends the
initial snapshot id to the Catalog, so the Catalog can validate the
operation.

Testing:
 - Added e2e tests.

Change-Id: I812d19736c8e563541b038970786de7710b59f31
Reviewed-on: http://gerrit.cloudera.org:8080/18648
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Handle concurrent Iceberg INSERT OVERWRITEs
> -------------------------------------------
>
>                 Key: IMPALA-11377
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11377
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Frontend
>    Affects Versions: Impala 4.1.0
>            Reporter: Tamas Mate
>            Assignee: Tamas Mate
>            Priority: Major
>              Labels: impala-iceberg
>
> As said in the parent ticket Iceberg now supports validations for concurrent writes through the ReplacePartitions API. This allows us to sort out the transactions during INSERT OVERWRITES.
> https://github.com/apache/iceberg/blob/9ab94f87de036c9cd91cf8353906a576b4a516ff/api/src/main/java/org/apache/iceberg/ReplacePartitions.java#L28-L34



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org