You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/10/24 10:22:45 UTC
[GitHub] [doris] huangkaiyan10 opened a new issue, #13603: at flinksql job，flink-doris-connector fail to SerializedThrowable
huangkaiyan10 opened a new issue, #13603:
URL: https://github.com/apache/doris/issues/13603
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
1.1.3
### What's Wrong?
kafka-2-doris-test-3-checkpoint-rocksdb-taskmanager-1-1 @ 100.81.116.55 (dataPort=43943).
org.apache.flink.util.SerializedThrowable: Task name with subtask : Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2 Failure reason: Task has failed.
at org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1389) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.lambda$triggerCheckpointBarrier$3(Task.java:1332) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836) ~[?:1.8.0_345]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811) ~[?:1.8.0_345]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_345]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_345]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1131) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:353) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_345]
Caused by: org.apache.flink.util.SerializedThrowable: Could not perform checkpoint 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2.
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1175) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 12 more
Caused by: org.apache.flink.util.SerializedThrowable: Could not complete snapshot 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2. Failure reason: Checkpoint was declined.
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:265) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:170) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:348) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.checkpointStreamOperator(RegularOperatorChain.java:233) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.buildOperatorSnapshotFutures(RegularOperatorChain.java:206) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.snapshotState(RegularOperatorChain.java:186) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:605) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1329) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1163) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 12 more
Caused by: org.apache.flink.util.SerializedThrowable
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:347) ~[?:1.8.0_345]
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323) ~[?:1.8.0_345]
at org.apache.doris.flink.sink.writer.DorisWriterStateSerializer.serialize(DorisWriterStateSerializer.java:42) ~[?:?]
at org.apache.doris.flink.sink.writer.DorisWriterStateSerializer.serialize(DorisWriterStateSerializer.java:31) ~[?:?]
at org.apache.flink.core.io.SimpleVersionedSerialization.writeVersionAndSerialize(SimpleVersionedSerialization.java:112) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.serialize(SimpleVersionedListState.java:102) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.serializeAll(SimpleVersionedListState.java:113) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.update(SimpleVersionedListState.java:71) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.operators.sink.StatefulSinkWriterStateHandler.snapshotState(StatefulSinkWriterStateHandler.java:106) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.operators.sink.SinkOperator.snapshotState(SinkOperator.java:156) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:219) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:170) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:348) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.checkpointStreamOperator(RegularOperatorChain.java:233) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.buildOperatorSnapshotFutures(RegularOperatorChain.java:206) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.snapshotState(RegularOperatorChain.java:186) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:605) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1329) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1163) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 12 more
2022-10-24 10:03:05,151 WARN org.apache.flink.runtime.checkpoint.CheckpointFailureManager [] - Failed to trigger or complete checkpoint 3 for job 29c9f4d44fab1b6d77f557fce62f5e74. (0 consecutive failed attempts so far)
org.apache.flink.runtime.checkpoint.CheckpointException: Task has failed.
at org.apache.flink.runtime.messages.checkpoint.SerializedCheckpointException.unwrap(SerializedCheckpointException.java:51) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveDeclineMessage(CheckpointCoordinator.java:988) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$declineCheckpoint$2(ExecutionGraphHandler.java:103) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$processCheckpointCoordinatorMessage$3(ExecutionGraphHandler.java:119) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_345]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_345]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_345]
Caused by: org.apache.flink.util.SerializedThrowable: Task name with subtask : Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2 Failure reason: Task has failed.
at org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1389) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.lambda$triggerCheckpointBarrier$3(Task.java:1332) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836) ~[?:1.8.0_345]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811) ~[?:1.8.0_345]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_345]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_345]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1131) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:353) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 1 more
Caused by: org.apache.flink.util.SerializedThrowable: Could not perform checkpoint 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2.
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1175) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:353) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 1 more
Caused by: org.apache.flink.util.SerializedThrowable: Could not complete snapshot 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2. Failure reason: Checkpoint was declined.
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:265) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:170) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:348) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.checkpointStreamOperator(RegularOperatorChain.java:233) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.buildOperatorSnapshotFutures(RegularOperatorChain.java:206) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.snapshotState(RegularOperatorChain.java:186) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:605) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1329) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1163) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:353) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 1 more
Caused by: org.apache.flink.util.SerializedThrowable
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:347) ~[?:1.8.0_345]
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323) ~[?:1.8.0_345]
at org.apache.doris.flink.sink.writer.DorisWriterStateSerializer.serialize(DorisWriterStateSerializer.java:42) ~[?:?]
at org.apache.doris.flink.sink.writer.DorisWriterStateSerializer.serialize(DorisWriterStateSerializer.java:31) ~[?:?]
at org.apache.flink.core.io.SimpleVersionedSerialization.writeVersionAndSerialize(SimpleVersionedSerialization.java:112) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.serialize(SimpleVersionedListState.java:102) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.serializeAll(SimpleVersionedListState.java:113) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.update(SimpleVersionedListState.java:71) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.operators.sink.StatefulSinkWriterStateHandler.snapshotState(StatefulSinkWriterStateHandler.java:106) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.operators.sink.SinkOperator.snapshotState(SinkOperator.java:156) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:219) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:170) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:348) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.checkpointStreamOperator(RegularOperatorChain.java:233) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.buildOperatorSnapshotFutures(RegularOperatorChain.java:206) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.snapshotState(RegularOperatorChain.java:186) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:605) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1329) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1163) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:353) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 1 more
2022-10-24 10:03:05,167 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1) (4b3d71d76e3eff70fdba661f49c1e8da) switched from RUNNING to FAILED on kafka-2-doris-test-3-checkpoint-rocksdb-taskmanager-1-1 @ 100.81.116.55 (dataPort=43943).
java.lang.Exception: Could not perform checkpoint 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2.
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1175) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:353) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345]
Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete snapshot 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2. Failure reason: Checkpoint was declined.
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:265) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:170) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:348) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.checkpointStreamOperator(RegularOperatorChain.java:233) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.buildOperatorSnapshotFutures(RegularOperatorChain.java:206) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.snapshotState(RegularOperatorChain.java:186) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:605) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1329) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1163) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 13 more
Caused by: java.lang.NullPointerException
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:347) ~[?:1.8.0_345]
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323) ~[?:1.8.0_345]
at org.apache.doris.flink.sink.writer.DorisWriterStateSerializer.serialize(DorisWriterStateSerializer.java:42) ~[?:?]
at org.apache.doris.flink.sink.writer.DorisWriterStateSerializer.serialize(DorisWriterStateSerializer.java:31) ~[?:?]
at org.apache.flink.core.io.SimpleVersionedSerialization.writeVersionAndSerialize(SimpleVersionedSerialization.java:112) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.serialize(SimpleVersionedListState.java:102) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.serializeAll(SimpleVersionedListState.java:113) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState.update(SimpleVersionedListState.java:71) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.operators.sink.StatefulSinkWriterStateHandler.snapshotState(StatefulSinkWriterStateHandler.java:106) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.operators.sink.SinkOperator.snapshotState(SinkOperator.java:156) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:219) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:170) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:348) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.checkpointStreamOperator(RegularOperatorChain.java:233) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.buildOperatorSnapshotFutures(RegularOperatorChain.java:206) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.snapshotState(RegularOperatorChain.java:186) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:605) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1329) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1315) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1163) ~[flink-dist_2.12-1.14.5.jar:1.14.5]
... 13 more
2022-10-24 10:03:05,171 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 29c9f4d44fab1b6d77f557fce62f5e74
2022-10-24 10:03:05,172 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0.
2022-10-24 10:03:05,172 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Removing registered reader after failure for subtask 0 of source Source: KafkaSource-default_catalog.default_database.kafka_table_1.
2022-10-24 10:03:05,173 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 1 tasks should be restarted to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0.
2022-10-24 10:03:05,173 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kafka-2-doris-test-3-checkpoint-rocksdb (29c9f4d44fab1b6d77f557fce62f5e74) switched from state RUNNING to RESTARTING.
2022-10-24 10:03:06,176 INFO org.apache.flink.runtime.executiongraph.ExecutionGr
### What You Expected?
flinksql set ：
-Dexecution.checkpointing.interval=2000
-Dstate.backend=rocksdb
-Dstate.checkpoints.dir=file:///opt/flink/checkpoints/
-Dstate.savepoints.dir=file:///opt/flink/savepoints/
flinksql ：
drop table IF EXISTS kafka_table_1;
CREATE TEMPORARY TABLE IF NOT EXISTS kafka_table_1 (
f_sequence INT,
f_random INT,
f_random_str STRING,
ts AS localtimestamp
) WITH (
'connector' = 'kafka',
'topic' = 'kafka_table_1',
'properties.bootstrap.servers' = '172.18.17.235:30997',
'properties.group.id' = 'streamx-testGroup',
'scan.startup.mode' = 'earliest-offset',
'format' = 'json'
);
drop table IF EXISTS cdc_doris_sink;
CREATE TEMPORARY TABLE IF NOT EXISTS cdc_doris_sink (f_sequence int, f_random int, f_random_str STRING) WITH (
'connector' = 'doris',
'fenodes' = '172.18.17.225:8030',
'table.identifier' = 'flink_doris_test.test_doris_1',
'username' = 'root',
'password' = 'mysql'
);
insert into
cdc_doris_sink
select
f_sequence,
f_random,
f_random_str
from
kafka_table_1;
i Expected kafka stream data can write doris
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org