You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Leoyzen (via GitHub)" <gi...@apache.org> on 2023/02/03 10:59:39 UTC
[GitHub] [hudi] Leoyzen commented on issue #7824: [SUPPORT] NPE occurs when enabling metadata on table which does'nt has metadata previously.
Leoyzen commented on issue #7824:
URL: https://github.com/apache/hudi/issues/7824#issuecomment-1415707154
@danny0405 No, the path doesn't exists.
After upgrading latest branch(0.13.0-rc1), the exception has changed.
```LOG
2023-02-03 18:54:52,537 [flink-akka.actor.default-dispatcher-39] INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Trying to recover from a global failure.
org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'bucket_write: tbl_resp_sink' (operator 2b7fd632fd7edc109c5698c0b534dcae).
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:597) ~[flink-dist-1.15-vvr-6.0.4-SNAPSHOT.jar:1.15-vvr-6.0.4-SNAPSHOT]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$start$0(StreamWriteOperatorCoordinator.java:196) ~[?:?]
at org.apache.hudi.sink.utils.NonThrownExecutor.handleException(NonThrownExecutor.java:142) ~[?:?]
at org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:133) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) ~[?:1.8.0_102]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) ~[?:1.8.0_102]
at java.lang.Thread.run(Thread.java:834) ~[?:1.8.0_102]
Caused by: org.apache.hudi.exception.HoodieException: Executor executes action [initialize instant 20230203185127227] error
... 6 more
Caused by: org.apache.hudi.exception.HoodieException: Failed to update metadata
at org.apache.hudi.client.HoodieFlinkTableServiceClient.writeTableMetadata(HoodieFlinkTableServiceClient.java:181) ~[?:?]
at org.apache.hudi.client.HoodieFlinkWriteClient.writeTableMetadata(HoodieFlinkWriteClient.java:279) ~[?:?]
at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:282) ~[?:?]
at org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:233) ~[?:?]
at org.apache.hudi.client.HoodieFlinkWriteClient.commit(HoodieFlinkWriteClient.java:111) ~[?:?]
at org.apache.hudi.client.HoodieFlinkWriteClient.commit(HoodieFlinkWriteClient.java:74) ~[?:?]
at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:199) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.doCommit(StreamWriteOperatorCoordinator.java:548) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:524) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:495) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$initInstant$6(StreamWriteOperatorCoordinator.java:413) ~[?:?]
at org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:130) ~[?:?]
... 3 more
Caused by: java.lang.IllegalStateException: Metadata table is not fully initialized yet.
at org.apache.hudi.common.util.ValidationUtils.checkState(ValidationUtils.java:67) ~[?:?]
at org.apache.hudi.metadata.FlinkHoodieBackedTableMetadataWriter.commit(FlinkHoodieBackedTableMetadataWriter.java:109) ~[?:?]
at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.processAndCommit(HoodieBackedTableMetadataWriter.java:823) ~[?:?]
at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:890) ~[?:?]
at org.apache.hudi.client.HoodieFlinkTableServiceClient.writeTableMetadata(HoodieFlinkTableServiceClient.java:179) ~[?:?]
at org.apache.hudi.client.HoodieFlinkWriteClient.writeTableMetadata(HoodieFlinkWriteClient.java:279) ~[?:?]
at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:282) ~[?:?]
at org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:233) ~[?:?]
at org.apache.hudi.client.HoodieFlinkWriteClient.commit(HoodieFlinkWriteClient.java:111) ~[?:?]
at org.apache.hudi.client.HoodieFlinkWriteClient.commit(HoodieFlinkWriteClient.java:74) ~[?:?]
at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:199) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.doCommit(StreamWriteOperatorCoordinator.java:548) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:524) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:495) ~[?:?]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$initInstant$6(StreamWriteOperatorCoordinator.java:413) ~[?:?]
at org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:130) ~[?:?]
... 3 more
```
It seems the procedure:
1. The hudi launched and found there isn't any metadata exists, try to create one.
2. The creation fails because of there is delta commit inflighting.
3. When the data flushing to the disk, the error occurs(the metadata has not been initilization).
```LOG
2023-02-03 18:54:08,449 [pool-24-thread-1] WARN org.apache.hudi.metadata.HoodieBackedTableMetadataWriter [] - Cannot initialize metadata table as operation(s) are in progress on the dataset: [[==>20230203184713442__deltacommit__INFLIGHT], [==>20230203185127185__rollback__INFLIGHT], [==>20230203185127227__deltacommit__INFLIGHT]]
..........
2023-02-03 18:54:08,238 [pool-24-thread-1] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata [] - Metadata table was not found at path oss://dengine-lake-zjk/cloudcode_pre/dwd_egc_adv_resp_intra/.hoodie/metadata
............
Caused by: java.lang.IllegalStateException: Metadata table is not fully initialized yet.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org