[GitHub] [iceberg] samredai commented on a change in pull request #4128: Core: Update SnapshotManager to use a transaction

[GitBox <gi...@apache.org>]

samredai commented on a change in pull request #4128:
URL: https://github.com/apache/iceberg/pull/4128#discussion_r807405529



##########
File path: core/src/main/java/org/apache/iceberg/CherryPickOperation.java
##########
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg;
+
+import java.util.List;
+import java.util.Map;
+import org.apache.iceberg.exceptions.CherrypickAncestorCommitException;
+import org.apache.iceberg.exceptions.ValidationException;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
+import org.apache.iceberg.util.PartitionSet;
+import org.apache.iceberg.util.PropertyUtil;
+import org.apache.iceberg.util.SnapshotUtil;
+import org.apache.iceberg.util.WapUtil;
+
+/**
+ * Cherry-picks or fast-forwards the current state to a snapshot.
+ * <p>
+ * This update is not exposed though the Table API. Instead, it is a package-private part of the Transaction API
+ * intended for use in {@link ManageSnapshots}.
+ */
+class CherryPickOperation extends MergingSnapshotProducer<CherryPickOperation> {
+
+  private final Map<Integer, PartitionSpec> specsById;
+  private Snapshot cherrypickSnapshot = null;
+  private boolean requireFastForward = false;
+  private PartitionSet replacedPartitions = null;
+
+  CherryPickOperation(String tableName, TableOperations ops) {
+    super(tableName, ops);
+    this.specsById = ops.current().specsById();
+  }
+
+  @Override
+  protected CherryPickOperation self() {
+    return this;
+  }
+
+  @Override
+  protected String operation() {
+    // snapshotOperation is used by SnapshotProducer when building and writing a new snapshot for cherrypick
+    Preconditions.checkNotNull(cherrypickSnapshot, "[BUG] Detected uninitialized operation");
+    return cherrypickSnapshot.operation();
+  }
+
+  public CherryPickOperation cherrypick(long snapshotId) {
+    TableMetadata current = current();
+    this.cherrypickSnapshot = current.snapshot(snapshotId);
+    ValidationException.check(cherrypickSnapshot != null,
+        "Cannot cherry-pick unknown snapshot ID: %s", snapshotId);
+
+    if (cherrypickSnapshot.operation().equals(DataOperations.APPEND)) {
+      // this property is set on target snapshot that will get published
+      String wapId = WapUtil.validateWapPublish(current, snapshotId);
+      if (wapId != null) {
+        set(SnapshotSummary.PUBLISHED_WAP_ID_PROP, wapId);
+      }
+
+      // link the snapshot about to be published on commit with the picked snapshot
+      set(SnapshotSummary.SOURCE_SNAPSHOT_ID_PROP, String.valueOf(snapshotId));
+
+      // Pick modifications from the snapshot
+      for (DataFile addedFile : cherrypickSnapshot.addedFiles()) {
+        add(addedFile);
+      }
+
+    } else if (cherrypickSnapshot.operation().equals(DataOperations.OVERWRITE) &&
+        PropertyUtil.propertyAsBoolean(cherrypickSnapshot.summary(), SnapshotSummary.REPLACE_PARTITIONS_PROP, false)) {
+      // the operation was ReplacePartitions. this can be cherry-picked iff the partitions have not been modified.
+      // detecting modification requires finding the new files since the parent was committed, so the parent must be an
+      // ancestor of the current state, or null if the overwrite was based on an empty table.
+      ValidationException.check(
+          cherrypickSnapshot.parentId() == null || isCurrentAncestor(current, cherrypickSnapshot.parentId()),
+          "Cannot cherry-pick overwrite not based on an ancestor of the current state: %s", snapshotId);
+
+      // this property is set on target snapshot that will get published
+      String wapId = WapUtil.validateWapPublish(current, snapshotId);
+      if (wapId != null) {
+        set(SnapshotSummary.PUBLISHED_WAP_ID_PROP, wapId);

Review comment:
       Will `wapId` stay as a snapshot property or will it eventually become a higher-level tag that references a transaction or branch?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org