[GitHub] [iceberg] ludlows commented on pull request #6760: Spark 3.3: use a deterministic where condition to make rewrite_data_files…

["ludlows (via GitHub)" <gi...@apache.org>]

ludlows commented on PR #6760:
URL: https://github.com/apache/iceberg/pull/6760#issuecomment-1431307623

   Hi @szehon-ho , thanks for your comments.
   I notice that the procedure `rewrite_data_files` first runs the function `checkAndApplyFilter` just before the `action.execute()`  https://github.com/apache/iceberg/blob/18d45b42b6504c03c71467a91ac5e67c9528f068/spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java#L120  
   
   However, if the where condition is always false like 
   `
   where=>'0=1'
   `
   the function `checkAndApplyFilter` will raise an IllegalArgumentException in line below.
   https://github.com/apache/iceberg/blob/18d45b42b6504c03c71467a91ac5e67c9528f068/spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java#L136
   
   to make the sql like `call catalog.system.rewrite_data_files(table=>'hive.tbl', where=>'0=1')` exit without exceptions, I proposed this PR. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org