[GitHub] [iceberg] karuppayya commented on a diff in pull request #7324: Honor spark case sensitivity in ALTER TABLE.. ORDERED BY

["karuppayya (via GitHub)" <gi...@apache.org>]

karuppayya commented on code in PR #7324:
URL: https://github.com/apache/iceberg/pull/7324#discussion_r1166062202


##########
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/SetWriteDistributionAndOrderingExec.scala:
##########
@@ -47,6 +48,7 @@ case class SetWriteDistributionAndOrderingExec(
         val txn = iceberg.table.newTransaction()
 
         val orderBuilder = txn.replaceSortOrder()
+        orderBuilder.caseSensitive(session.conf.get(SQLConf.CASE_SENSITIVE, SQLConf.CASE_SENSITIVE.defaultValue.get))

Review Comment:
   done



##########
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/SetWriteDistributionAndOrderingExec.scala:
##########
@@ -47,6 +48,7 @@ case class SetWriteDistributionAndOrderingExec(
         val txn = iceberg.table.newTransaction()
 
         val orderBuilder = txn.replaceSortOrder()

Review Comment:
   done



##########
api/src/main/java/org/apache/iceberg/SortOrderBuilder.java:
##########
@@ -103,4 +103,12 @@ default R desc(Term term) {
    * @return this for method chaining
    */
   R desc(Term term, NullOrder nullOrder);
+
+  /**
+   * Set case sensitivity of sort column name resolution.
+   *
+   * @param caseSensitive when true, column name resolution is case-sensitive
+   * @return this for method chaining
+   */
+  R caseSensitive(boolean caseSensitive);

Review Comment:
   done



##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSetWriteDistributionAndOrdering.java:
##########
@@ -64,6 +67,25 @@ public void testSetWriteOrderByColumn() {
     Assert.assertEquals("Should have expected order", expected, table.sortOrder());
   }
 
+  @Test
+  public void testSetWriteOrderWithCaseSensitiveColumnNames() {
+    sql(
+        "CREATE TABLE %s (Id bigint NOT NULL, Category string, ts timestamp, data string) USING iceberg",
+        tableName);
+    Table table = validationCatalog.loadTable(tableIdent);
+    Assert.assertTrue("Table should start unsorted", table.sortOrder().isUnsorted());
+    Assertions.assertThatThrownBy(() -> {
+      sql(String.format("SET %s=true", SQLConf.CASE_SENSITIVE().key()));

Review Comment:
   Fixed the `sql` and move the set commands outside



##########
core/src/test/java/org/apache/iceberg/TestSortOrder.java:
##########
@@ -365,4 +367,26 @@ public void testPreservingOrderSortedColumnNames() {
     Set<String> sortedCols = SortOrderUtil.orderPreservingSortedColumns(order);
     Assert.assertEquals(ImmutableSet.of("data"), sortedCols);
   }
+
+  @Test
+  public void testCaseSensitiveSortedColumnNames() {

Review Comment:
   There are no tests currently for testing the caseSensitive method. Shall I remove it from here and add in a dfif pr?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org