You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Sagar Sumit (Jira)" <ji...@apache.org> on 2023/03/29 10:42:00 UTC
[jira] [Closed] (HUDI-4182) HoodieIndexer failing when trying to invoke it as Single Writer
[ https://issues.apache.org/jira/browse/HUDI-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sagar Sumit closed HUDI-4182.
-----------------------------
Resolution: Fixed
> HoodieIndexer failing when trying to invoke it as Single Writer
> ---------------------------------------------------------------
>
> Key: HUDI-4182
> URL: https://issues.apache.org/jira/browse/HUDI-4182
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Alexey Kudinkin
> Assignee: Sagar Sumit
> Priority: Critical
>
> Trying to invoke Async Indexer with following properties fails:
> {code:java}
> hoodie.metadata.enable=true
> hoodie.metadata.index.async=true
> hoodie.metadata.index.column.stats.enable=true
> hoodie.write.concurrency.mode=single_writer
> hoodie.cleaner.policy.failed.writes=EAGER {code}
> {code:java}
> 2022-06-02 21:51:58,123 ERROR utilities.UtilHelpers: Indexer failed
> org.apache.hudi.exception.HoodieIndexException: Need to set hoodie.write.concurrency.mode as OPTIMISTIC_CONCURRENCY_CONTROL and configure lock provider class
> at org.apache.hudi.table.action.index.ScheduleIndexActionExecutor.validateBeforeScheduling(ScheduleIndexActionExecutor.java:137)
> at org.apache.hudi.table.action.index.ScheduleIndexActionExecutor.execute(ScheduleIndexActionExecutor.java:84)
> at org.apache.hudi.table.HoodieSparkCopyOnWriteTable.scheduleIndexing(HoodieSparkCopyOnWriteTable.java:286)
> at org.apache.hudi.client.BaseHoodieWriteClient.scheduleIndexing(BaseHoodieWriteClient.java:1016)
> at org.apache.hudi.utilities.HoodieIndexer.doSchedule(HoodieIndexer.java:234)
> at org.apache.hudi.utilities.HoodieIndexer.scheduleAndRunIndexing(HoodieIndexer.java:276)
> at org.apache.hudi.utilities.HoodieIndexer.lambda$start$1(HoodieIndexer.java:198)
> at org.apache.hudi.utilities.UtilHelpers.retry(UtilHelpers.java:541)
> at org.apache.hudi.utilities.HoodieIndexer.start(HoodieIndexer.java:185)
> at org.apache.hudi.utilities.HoodieIndexer.main(HoodieIndexer.java:154)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)