You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "RussellSpitzer (via GitHub)" <gi...@apache.org> on 2023/07/02 01:57:32 UTC

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #7931: Hive: Set commit state as Unknown before throwing CommitStateUnknownException

RussellSpitzer commented on code in PR #7931:
URL: https://github.com/apache/iceberg/pull/7931#discussion_r1249086717


##########
hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java:
##########
@@ -352,6 +352,49 @@ public void testNoLockThriftExceptionConcurrentCommit() throws TException, Inter
     Assert.assertEquals("New metadata files should not exist", 2, metadataFileCount(ops.current()));
   }
 
+  @Test
+  public void testLockExceptionUnknownSuccessCommit() throws TException, InterruptedException {
+    Table table = catalog.loadTable(TABLE_IDENTIFIER);
+    HiveTableOperations ops = (HiveTableOperations) ((HasTableOperations) table).operations();
+
+    TableMetadata metadataV1 = ops.current();
+
+    table.updateSchema().addColumn("n", Types.IntegerType.get()).commit();
+
+    ops.refresh();
+
+    TableMetadata metadataV2 = ops.current();
+
+    Assertions.assertThat(ops.current().schema().columns().size()).isEqualTo(2);
+
+    HiveTableOperations spyOps = spy(ops);
+
+    // Simulate a communication error after a successful commit
+    doAnswer(
+            i -> {
+              org.apache.hadoop.hive.metastore.api.Table tbl =
+                  i.getArgument(0, org.apache.hadoop.hive.metastore.api.Table.class);
+              String location = i.getArgument(2, String.class);
+              ops.persistTable(tbl, true, location);
+              throw new LockException("Datacenter on fire");
+            })
+        .when(spyOps)
+        .persistTable(any(), anyBoolean(), any());
+
+    Assertions.assertThatThrownBy(() -> spyOps.commit(metadataV2, metadataV1))
+        .hasMessageContaining("Failed to heartbeat for hive lock while")
+        .isInstanceOf(CommitStateUnknownException.class);
+
+    ops.refresh();
+
+    Assertions.assertThat(ops.current())
+        .as("Current metadata should have changed")

Review Comment:
   Hmm, for now could we just check locations? Then we can do a full equal implementation in another or



##########
hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java:
##########
@@ -352,6 +352,49 @@ public void testNoLockThriftExceptionConcurrentCommit() throws TException, Inter
     Assert.assertEquals("New metadata files should not exist", 2, metadataFileCount(ops.current()));
   }
 
+  @Test
+  public void testLockExceptionUnknownSuccessCommit() throws TException, InterruptedException {
+    Table table = catalog.loadTable(TABLE_IDENTIFIER);
+    HiveTableOperations ops = (HiveTableOperations) ((HasTableOperations) table).operations();
+
+    TableMetadata metadataV1 = ops.current();
+
+    table.updateSchema().addColumn("n", Types.IntegerType.get()).commit();
+
+    ops.refresh();
+
+    TableMetadata metadataV2 = ops.current();
+
+    Assertions.assertThat(ops.current().schema().columns().size()).isEqualTo(2);
+
+    HiveTableOperations spyOps = spy(ops);
+
+    // Simulate a communication error after a successful commit
+    doAnswer(
+            i -> {
+              org.apache.hadoop.hive.metastore.api.Table tbl =
+                  i.getArgument(0, org.apache.hadoop.hive.metastore.api.Table.class);
+              String location = i.getArgument(2, String.class);
+              ops.persistTable(tbl, true, location);
+              throw new LockException("Datacenter on fire");
+            })
+        .when(spyOps)
+        .persistTable(any(), anyBoolean(), any());
+
+    Assertions.assertThatThrownBy(() -> spyOps.commit(metadataV2, metadataV1))
+        .hasMessageContaining("Failed to heartbeat for hive lock while")
+        .isInstanceOf(CommitStateUnknownException.class);
+
+    ops.refresh();
+
+    Assertions.assertThat(ops.current())
+        .as("Current metadata should have changed")

Review Comment:
   Hmm, for now could we just check locations? Then we can do a full equal implementation in another PR



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org