[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6760: Spark 3.3: use a deterministic where condition to make rewrite_data_files…

["szehon-ho (via GitHub)" <gi...@apache.org>]

szehon-ho commented on code in PR #6760:
URL: https://github.com/apache/iceberg/pull/6760#discussion_r1133031390


##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java:
##########
@@ -130,9 +131,10 @@ private RewriteDataFiles checkAndApplyFilter(
       RewriteDataFiles action, String where, String tableName) {
     if (where != null) {
       try {
-        Expression expression =
-            SparkExpressionConverter.collectResolvedSparkExpression(spark(), tableName, where);
-        return action.filter(SparkExpressionConverter.convertToIcebergExpression(expression));
+        Option<Expression> expressionOption =
+                SparkExpressionConverter.collectResolvedSparkExpressionOption(spark(), tableName, where);
+        if (expressionOption.isEmpty()) return action.filter(Expressions.alwaysFalse());

Review Comment:
   Nit: I'm not sure if checkstyle/spotless will fail here, but think we need extra newline in any case for the return.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org