[GitHub] [hudi] kazdy commented on a diff in pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables

["kazdy (via GitHub)" <gi...@apache.org>]

kazdy commented on code in PR #9277:
URL: https://github.com/apache/hudi/pull/9277#discussion_r1276494722


##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/UpdateHoodieTableCommand.scala:
##########
@@ -31,6 +34,22 @@ import org.apache.spark.sql.hudi.ProvidesHoodieConfig
 case class UpdateHoodieTableCommand(ut: UpdateTable) extends HoodieLeafRunnableCommand
   with SparkAdapterSupport with ProvidesHoodieConfig {
 
+  private def buildUpdateConfig( sparkSession: SparkSession,
+                                    hoodieCatalogTable: HoodieCatalogTable): Map[String, String] = {
+    val optimizedWrite = sparkSession.sqlContext.conf.getConfString(SPARK_SQL_OPTIMIZED_WRITES.key()
+      , SPARK_SQL_OPTIMIZED_WRITES.defaultValue()) == "true"
+    val shouldCombine = !StringUtils.isNullOrEmpty(hoodieCatalogTable.preCombineKey.getOrElse(""))
+
+    if (!shouldCombine && hoodieCatalogTable.tableType.name().equals("MERGE_ON_READ")) {
+      throw new HoodieException("Precombine field must be set for MERGE_ON_READ table type to execute UPDATE statement.")

Review Comment:
   @danny0405  Yes, this is intentional as for MoR it is a requirement to have precombine field (but it's not enforced yet atm), see: https://github.com/apache/hudi/pull/9031
   
   In theory, once #9091 is merged this exception should never be thrown for new tables, but for old tables it's better to guard against this.
   I decided to throw this exception here since different record impl throw different exceptions for MoR when there's no precombine field -  "Failed to upsert for commit" or "(Part -) not found, acceptable fields were: ..." it was hard to test for this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org