[GitHub] [iceberg] szehon-ho commented on a diff in pull request #7324: Honor spark case sensitivity in ALTER TABLE.. ORDERED BY

["szehon-ho (via GitHub)" <gi...@apache.org>]

szehon-ho commented on code in PR #7324:
URL: https://github.com/apache/iceberg/pull/7324#discussion_r1163395482


##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSetWriteDistributionAndOrdering.java:
##########
@@ -64,6 +67,25 @@ public void testSetWriteOrderByColumn() {
     Assert.assertEquals("Should have expected order", expected, table.sortOrder());
   }
 
+  @Test
+  public void testSetWriteOrderWithCaseSensitiveColumnNames() {
+    sql(
+        "CREATE TABLE %s (Id bigint NOT NULL, Category string, ts timestamp, data string) USING iceberg",
+        tableName);
+    Table table = validationCatalog.loadTable(tableIdent);
+    Assert.assertTrue("Table should start unsorted", table.sortOrder().isUnsorted());
+    Assertions.assertThatThrownBy(() -> {
+      sql(String.format("SET %s=true", SQLConf.CASE_SENSITIVE().key()));

Review Comment:
   Doesn't sql() already do format?



##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSetWriteDistributionAndOrdering.java:
##########
@@ -64,6 +67,25 @@ public void testSetWriteOrderByColumn() {
     Assert.assertEquals("Should have expected order", expected, table.sortOrder());
   }
 
+  @Test
+  public void testSetWriteOrderWithCaseSensitiveColumnNames() {
+    sql(
+        "CREATE TABLE %s (Id bigint NOT NULL, Category string, ts timestamp, data string) USING iceberg",
+        tableName);
+    Table table = validationCatalog.loadTable(tableIdent);
+    Assert.assertTrue("Table should start unsorted", table.sortOrder().isUnsorted());
+    Assertions.assertThatThrownBy(() -> {
+      sql(String.format("SET %s=true", SQLConf.CASE_SENSITIVE().key()));

Review Comment:
   Should we also move the set outside?  So that assertThatThrownBy is only about the alter stmt?



##########
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/SetWriteDistributionAndOrderingExec.scala:
##########
@@ -47,6 +48,7 @@ case class SetWriteDistributionAndOrderingExec(
         val txn = iceberg.table.newTransaction()
 
         val orderBuilder = txn.replaceSortOrder()
+        orderBuilder.caseSensitive(session.conf.get(SQLConf.CASE_SENSITIVE, SQLConf.CASE_SENSITIVE.defaultValue.get))

Review Comment:
   nit: can this be on same builder?
   
   ```
   orderBuilder = txn.replaceSortOrder()
      .caseSensitive(...)
   ```



##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSetWriteDistributionAndOrdering.java:
##########
@@ -64,6 +67,25 @@ public void testSetWriteOrderByColumn() {
     Assert.assertEquals("Should have expected order", expected, table.sortOrder());
   }
 
+  @Test
+  public void testSetWriteOrderWithCaseSensitiveColumnNames() {
+    sql(
+        "CREATE TABLE %s (Id bigint NOT NULL, Category string, ts timestamp, data string) USING iceberg",
+        tableName);
+    Table table = validationCatalog.loadTable(tableIdent);
+    Assert.assertTrue("Table should start unsorted", table.sortOrder().isUnsorted());
+    Assertions.assertThatThrownBy(() -> {
+      sql(String.format("SET %s=true", SQLConf.CASE_SENSITIVE().key()));
+      sql("ALTER TABLE %s WRITE ORDERED BY category, id", tableName);
+    }).isInstanceOf(ValidationException.class);
+
+    Assertions.assertThatNoException()
+        .isThrownBy(() -> {
+          sql(String.format("SET %s=false", SQLConf.CASE_SENSITIVE().key()));

Review Comment:
   Same as above



##########
core/src/test/java/org/apache/iceberg/TestSortOrder.java:
##########
@@ -365,4 +367,26 @@ public void testPreservingOrderSortedColumnNames() {
     Set<String> sortedCols = SortOrderUtil.orderPreservingSortedColumns(order);
     Assert.assertEquals(ImmutableSet.of("data"), sortedCols);
   }
+
+  @Test
+  public void testCaseSensitiveSortedColumnNames() {

Review Comment:
   Not sure this test is testing any new code?  If we want an API level test, shouldnt it be for testing BaseReplaceSortOrder (which is the new code)?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org