[I] Spark2 bundles made from master after 2022-07-23 failed to stop [hudi]

["hudi-bot (via GitHub)" <gi...@apache.org>]

hudi-bot opened a new issue, #15398:
URL: https://github.com/apache/hudi/issues/15398

   Daily Spark2 Structured streaming application built against the latest change on master started failing on 2022-07-23 with the error:
   {quote}
   22/07/28 10:52:50 ERROR org.apache.hudi.HoodieStreamingSink: Micro batch id=2 threw following exception: 
   org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'test' not found;
   {quote}
   Actually the database exists (created by the Hudi). Spark3 application with the same code works fine.
   Hudi options used for writing:
   {code}
    Map("hoodie.datasource.write.operation" -> "upsert",
     "hoodie.datasource.write.table.type" -> "MERGE_ON_READ",
     "hoodie.datasource.write.recordkey.field" -> s"$Meta.$Key",
     "hoodie.datasource.write.precombine.field" -> s"$Meta.$Version",
     "hoodie.datasource.write.payload.class" -> "com.walmart.udps.hudi.integration.HudiTestAvroPayload",
     "hoodie.compaction.payload.class" -> "com.walmart.udps.hudi.integration.HudiTestAvroPayload",
     "hoodie.datasource.write.partitionpath.field" -> TopLevelPartition,
     "hoodie.datasource.write.hive_style_partitioning" -> "true",
     "hoodie.datasource.hive_sync.enable" -> "true",
     "hoodie.datasource.hive_sync.mode" -> "hms",
     "hoodie.datasource.hive_sync.database" -> dbName(),
     "hoodie.datasource.hive_sync.partition_fields" -> TopLevelPartition,
     "hoodie.datasource.compaction.async.enable" -> "false"
   )
   {code}
   Cloud platform - Google Dataproc. 
   Spark version - 3.1.3.
   
   ## JIRA info
   
   - Link: https://issues.apache.org/jira/browse/HUDI-4735
   - Type: Bug
   - Fix version(s):
     - 1.1.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Spark2 bundles made from master after 2022-07-23 failed to stop [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on issue #15398:
URL: https://github.com/apache/hudi/issues/15398#issuecomment-4795361071

   Triage update: Spark2 support (and Scala 2.11) was fully removed from Hudi in commit 117d8227d44b (HUDI-8089, PR #11788), with residual cleanup in b6aed1eb49ce (PR #12079). Current master only ships bundles for Spark 3.3.x, 3.4.x, 3.5.x, 4.0.x, and 4.1.x — there is no longer a `hudi-spark2*` module or Spark2 bundle. The reporter's environment (Spark 3.1.3 / Spark2 bundle) is also no longer a supported configuration. Closing as fixed/obsolete; please reopen against a supported Spark 3.5+ or 4.x bundle if the issue still reproduces there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Spark2 bundles made from master after 2022-07-23 failed to stop [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan closed issue #15398: Spark2 bundles made from master after 2022-07-23 failed to stop
URL: https://github.com/apache/hudi/issues/15398


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org