[jira] [Updated] (HUDI-4735) Spark2 bundles made from master after 2022-07-23 failed to stop

["Zhaojing Yu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhaojing Yu updated HUDI-4735:
------------------------------
    Fix Version/s: 0.13.0
                       (was: 0.12.1)

> Spark2 bundles made from master after 2022-07-23 failed to stop
> ---------------------------------------------------------------
>
>                 Key: HUDI-4735
>                 URL: https://issues.apache.org/jira/browse/HUDI-4735
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: hive, spark, spark-sql
>            Reporter: Vitali Obukh
>            Assignee: Raymond Xu
>            Priority: Major
>             Fix For: 0.13.0
>
>
> Daily Spark2 Structured streaming application built against the latest change on master started failing on 2022-07-23 with the error:
> {quote}
> 22/07/28 10:52:50 ERROR org.apache.hudi.HoodieStreamingSink: Micro batch id=2 threw following exception: 
> org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'test' not found;
> {quote}
> Actually the database exists (created by the Hudi). Spark3 application with the same code works fine.
> Hudi options used for writing:
> {code}
>  Map("hoodie.datasource.write.operation" -> "upsert",
>   "hoodie.datasource.write.table.type" -> "MERGE_ON_READ",
>   "hoodie.datasource.write.recordkey.field" -> s"$Meta.$Key",
>   "hoodie.datasource.write.precombine.field" -> s"$Meta.$Version",
>   "hoodie.datasource.write.payload.class" -> "com.walmart.udps.hudi.integration.HudiTestAvroPayload",
>   "hoodie.compaction.payload.class" -> "com.walmart.udps.hudi.integration.HudiTestAvroPayload",
>   "hoodie.datasource.write.partitionpath.field" -> TopLevelPartition,
>   "hoodie.datasource.write.hive_style_partitioning" -> "true",
>   "hoodie.datasource.hive_sync.enable" -> "true",
>   "hoodie.datasource.hive_sync.mode" -> "hms",
>   "hoodie.datasource.hive_sync.database" -> dbName(),
>   "hoodie.datasource.hive_sync.partition_fields" -> TopLevelPartition,
>   "hoodie.datasource.compaction.async.enable" -> "false"
> )
> {code}
> Cloud platform - Google Dataproc. 
> Spark version - 3.1.3.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)