You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 06:10:26 UTC
[I] Spark SQL MERGE INTO statement should always do upsert if there's matching update clause [hudi]
hudi-bot opened a new issue, #15607:
URL: https://github.com/apache/hudi/issues/15607
h4. *UPDATED*
Aforementioned issue was actually a result of misconfiguration of the Merge Into statement – MIT was using "insert" operation instead of "upsert".
Real issue though is that MIT implicitly predicates using "upsert" operation onto whether "preCombine" config is set. Instead, it should always specify operation as "upsert", since MIT allows to specify updating semantics w/o requiring presence of the "preCombine" field
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5324
- Type: Improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Spark SQL MERGE INTO statement should always do upsert if there's matching update clause [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15607:
URL: https://github.com/apache/hudi/issues/15607#issuecomment-4795452948
This appears to be **fixed in master**.
`MergeIntoHoodieTableCommand.getOperationType` no longer keys the choice off `preCombine` alone — it now also checks `updatingActions.isEmpty`, so any MERGE INTO statement with a matched `UPDATE` clause picks `UPSERT` regardless of whether an ordering/preCombine field is configured:
```scala
private def getOperationType(parameters: Map[String, String]) = {
if (StringUtils.isNullOrEmpty(ConfigUtils.getOrderingFieldsStrDuringWrite(parameters.asJava)) && updatingActions.isEmpty) {
INSERT_OPERATION_OPT_VAL
} else {
UPSERT_OPERATION_OPT_VAL
}
}
```
(`hudi-spark-datasource/hudi-spark3-common/.../MergeIntoHoodieTableCommand.scala` line 538, mirrored in the spark4-common variant.)
The `updatingActions.isEmpty` clause was added in commit [`0ca0999`](https://github.com/apache/hudi/commit/0ca0999420ca36b777b23fc9cf3998ce8f31481f) (HUDI-6464, #9083). Closing as fixed — please reopen if you can reproduce on current master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Spark SQL MERGE INTO statement should always do upsert if there's matching update clause [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15607: Spark SQL MERGE INTO statement should always do upsert if there's matching update clause
URL: https://github.com/apache/hudi/issues/15607
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org