[jira] [Created] (HUDI-5150) False write success status on some errors

["Victor Sadkov (Jira)" <ji...@apache.org>]

Victor Sadkov created HUDI-5150:
-----------------------------------

             Summary: False write success status on some errors
                 Key: HUDI-5150
                 URL: https://issues.apache.org/jira/browse/HUDI-5150
             Project: Apache Hudi
          Issue Type: Bug
          Components: core
            Reporter: Victor Sadkov


HoodieSparkSqlWriter [returns a success status|https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L337] in its {{write()}} method instead of throwing an exception when something fails (and such an error can legitimately happen, for example [here|https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L767]).

And there is a multitude of places where return values of this method are ignored (unit tests were omitted, but there is a lot of such cases over there as well):


[https://github.com/apache/hudi/blob/master/rfc/rfc-38/rfc-38.md#L77]

[https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala#L351]

[https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala#L148]

[https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieStreamingSink.scala#L128]

[https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableDropPartitionCommand.scala#L62]

[https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/TruncateHoodieTableCommand.scala#L88]



This makes Hudi to swallow an error and report being successful on some write failures.

This issue actually happened in our infrastructure, but I don't have a good example to reproduce it, unfortunately.

For me it feels like a critical issue, please adjust the priority accordingly if I was not right :)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)