You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "weijietong (via GitHub)" <gi...@apache.org> on 2026/08/21 03:53:47 UTC

[PR] to DE insert into path,the target table should also be target splits [paimon]

weijietong opened a new pull request, #9333:
URL: https://github.com/apache/paimon/pull/9333

   ### Purpose
   
   to DE table, the insert path's left anti join could be more efficence by making the target table be target spltis
   
   ### Tests
   
   minor change, no tests,the existing one cove


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark]to DE insert into path,the target table should also be target splits [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #9333:
URL: https://github.com/apache/paimon/pull/9333


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark]to DE insert into path,the target table should also be target splits [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #9333:
URL: https://github.com/apache/paimon/pull/9333#discussion_r3827438647


##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/commands/MergeIntoPaimonDataEvolutionTable.scala:
##########
@@ -837,7 +837,7 @@ case class MergeIntoPaimonDataEvolutionTable(
       mergeFields.filter(field => targetTable.output.exists(attr => attr.equals(field)))
 
     val targetReadPlan =
-      touchedFileTargetRelation.copy(targetRelation.table, allReadFieldsOnTarget.toSeq)
+      touchedFileTargetRelation.copy(output = allReadFieldsOnTarget.toSeq)

Review Comment:
   Could you apply the same change to paimon-spark/paimon-spark-4.0/src/main/scala/org/apache/paimon/spark/commands/MergeIntoPaimonDataEvolutionTable.scala? Spark 4.0 compiles its own copy of this class, and its insertActionInvoke still calls touchedFileTargetRelation.copy(targetRelation.table, ...). That replaces the KnownSplitsTable with the original target table, so WHEN NOT MATCHED continues to scan the full target table on Spark 4.0. A regression assertion on the insert-side scan or file count would also prevent the two implementations from drifting again.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark]to DE insert into path,the target table should also be target splits [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #9333:
URL: https://github.com/apache/paimon/pull/9333#issuecomment-5365930143

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org