[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6633: Spark 3.3: Fix predicate pushdown for copy-on-write MERGE commands

["aokolnychyi (via GitHub)" <gi...@apache.org>]

aokolnychyi commented on code in PR #6633:
URL: https://github.com/apache/iceberg/pull/6633#discussion_r1082998249


##########
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteMergeIntoTable.scala:
##########
@@ -187,14 +187,12 @@ object RewriteMergeIntoTable extends RewriteRowLevelIcebergCommand with Predicat
     val readRelation = buildRelationWithAttrs(relation, operationTable, metadataAttrs)
     val readAttrs = readRelation.output
 
-    val (targetCond, joinCond) = splitMergeCond(cond, readRelation)

Review Comment:
   I reverted changes in #6534 for copy-on-write operations. It was not safe as pushing the join condition into a filter on the left side is not safe in LeftOuter and FullOuter joins. It changes the output, which can lead to loosing records that did not match the condition.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org