You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "zxylmqx (via GitHub)" <gi...@apache.org> on 2023/05/29 08:02:35 UTC
[GitHub] [seatunnel] zxylmqx commented on issue #4838: 当我使用zeta引擎时提交第二个cdc任务会停止所有cdc任务
zxylmqx commented on issue #4838:
URL: https://github.com/apache/seatunnel/issues/4838#issuecomment-1566723445
已经指定了不同的[sink.label.prefix]，而且这个是传入的变量，保证每次启动都是唯一的，目前将checkpoint从hdfs变成了locol，错误变成了
2023-05-29 15:07:42,928 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] turn from state DEPLOYING to RUNNING.
2023-05-29 15:07:43,010 ERROR org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - task running failed
java.util.concurrent.CompletionException: org.apache.seatunnel.common.utils.SeaTunnelException: java.io.EOFException
at com.hazelcast.spi.impl.AbstractInvocationFuture.returnOrThrowWithJoinConventions(AbstractInvocationFuture.java:819) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.AbstractInvocationFuture.resolveAndThrowWithJoinConvention(AbstractInvocationFuture.java:835) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.AbstractInvocationFuture.join(AbstractInvocationFuture.java:553) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.restoreTaskState(CheckpointCoordinator.java:250) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$1(CheckpointCoordinator.java:189) ~[seatunnel-starter.jar:2.3.1]
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640) ~[?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352]
Caused by: org.apache.seatunnel.common.utils.SeaTunnelException: java.io.EOFException
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:100) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.1]
Caused by: java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197) ~[?:1.8.0_352]
at java.io.DataInputStream.readLong(DataInputStream.java:416) ~[?:1.8.0_352]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:45) ~[?:?]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:29) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$null$2(SinkFlowLifeCycle.java:239) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.common.utils.ExceptionUtil.sneaky(ExceptionUtil.java:138) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$restoreState$3(SinkFlowLifeCycle.java:235) ~[seatunnel-starter.jar:2.3.1]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_352]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_352]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_352]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_352]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:240) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$14(SeaTunnelTask.java:383) ~[seatunnel-starter.jar:2.3.1]
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_352]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_352]
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:1.8.0_352]
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) ~[?:1.8.0_352]
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:380) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:96) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.1]
2023-05-29 15:07:43,010 INFO org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-05-29 15:07:43,010 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] checkpoint have error, cancel the pipeline
2023-05-29 15:07:43,011 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,014 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,014 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,015 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] operator to member [bigdata07]:5801
2023-05-29 15:07:43,015 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,018 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] operator to member [bigdata07]:5801
2023-05-29 15:07:43,019 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] operator to member [bigdata07]:5801
2023-05-29 15:07:43,019 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn to end state CANCELED.
2023-05-29 15:07:43,019 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn to end state CANCELED.
2023-05-29 15:07:43,019 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] end with state CANCELED
2023-05-29 15:07:43,019 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] end with state CANCELED
2023-05-29 15:07:43,021 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn to end state CANCELED.
2023-05-29 15:07:43,021 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] end with state CANCELED
2023-05-29 15:07:43,022 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] end with state FAILED
2023-05-29 15:07:43,022 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Can restore pipeline Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)]
2023-05-29 15:07:43,022 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Restore time 2, pipeline Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)]
2023-05-29 15:07:43,023 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn to state CREATED.
2023-05-29 15:07:43,024 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn to state CREATED.
2023-05-29 15:07:43,024 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn to state CREATED.
2023-05-29 15:07:43,025 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] is in state CREATED when init state future
2023-05-29 15:07:43,025 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] is in state CREATED when init state future
2023-05-29 15:07:43,025 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] is in state CREATED when init state future
2023-05-29 15:07:43,025 INFO org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - received restore CheckpointCoordinator with alreadyStarted= false
2023-05-29 15:07:43,025 INFO org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start clean pending checkpoint cause CheckpointCoordinator reset.
2023-05-29 15:07:43,025 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] turn from state CREATED to SCHEDULED.
2023-05-29 15:07:43,026 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received slot active check success, profile: SlotProfile{worker=[bigdata07]:5801, slotID=1, ownerJobID=715450079284035585, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'}
2023-05-29 15:07:43,026 INFO org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler - use active old profile: SlotProfile{worker=[bigdata07]:5801, slotID=1, ownerJobID=715450079284035585, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'} for task Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)]
2023-05-29 15:07:43,026 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state CREATED to SCHEDULED.
2023-05-29 15:07:43,027 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received slot active check success, profile: SlotProfile{worker=[bigdata07]:5801, slotID=2, ownerJobID=715450079284035585, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'}
2023-05-29 15:07:43,027 INFO org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler - use active old profile: SlotProfile{worker=[bigdata07]:5801, slotID=2, ownerJobID=715450079284035585, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'} for task Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)]
2023-05-29 15:07:43,028 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state CREATED to SCHEDULED.
2023-05-29 15:07:43,028 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received slot active check success, profile: SlotProfile{worker=[bigdata07]:5801, slotID=3, ownerJobID=715450079284035585, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'}
2023-05-29 15:07:43,028 INFO org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler - use active old profile: SlotProfile{worker=[bigdata07]:5801, slotID=3, ownerJobID=715450079284035585, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'} for task Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)]
2023-05-29 15:07:43,028 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state CREATED to SCHEDULED.
2023-05-29 15:07:43,029 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] turn from state SCHEDULED to DEPLOYING.
2023-05-29 15:07:43,030 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state SCHEDULED to DEPLOYING.
2023-05-29 15:07:43,031 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state SCHEDULED to DEPLOYING.
2023-05-29 15:07:43,032 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state SCHEDULED to DEPLOYING.
2023-05-29 15:07:43,066 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state DEPLOYING to RUNNING.
2023-05-29 15:07:43,073 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state DEPLOYING to RUNNING.
2023-05-29 15:07:43,074 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state DEPLOYING to RUNNING.
2023-05-29 15:07:43,074 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] turn from state DEPLOYING to RUNNING.
2023-05-29 15:07:43,151 ERROR org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - task running failed
java.util.concurrent.CompletionException: org.apache.seatunnel.common.utils.SeaTunnelException: java.io.EOFException
at com.hazelcast.spi.impl.AbstractInvocationFuture.returnOrThrowWithJoinConventions(AbstractInvocationFuture.java:819) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.AbstractInvocationFuture.resolveAndThrowWithJoinConvention(AbstractInvocationFuture.java:835) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.AbstractInvocationFuture.join(AbstractInvocationFuture.java:553) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.restoreTaskState(CheckpointCoordinator.java:250) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$1(CheckpointCoordinator.java:189) ~[seatunnel-starter.jar:2.3.1]
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640) ~[?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352]
Caused by: org.apache.seatunnel.common.utils.SeaTunnelException: java.io.EOFException
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:100) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.1]
Caused by: java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197) ~[?:1.8.0_352]
at java.io.DataInputStream.readLong(DataInputStream.java:416) ~[?:1.8.0_352]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:45) ~[?:?]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:29) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$null$2(SinkFlowLifeCycle.java:239) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.common.utils.ExceptionUtil.sneaky(ExceptionUtil.java:138) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$restoreState$3(SinkFlowLifeCycle.java:235) ~[seatunnel-starter.jar:2.3.1]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_352]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_352]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_352]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_352]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:240) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$14(SeaTunnelTask.java:383) ~[seatunnel-starter.jar:2.3.1]
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_352]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_352]
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:1.8.0_352]
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:1.8.0_352]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_352]
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) ~[?:1.8.0_352]
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:380) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:96) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48) ~[seatunnel-starter.jar:2.3.1]
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123) ~[seatunnel-starter.jar:2.3.1]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.1]
2023-05-29 15:07:43,151 INFO org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-05-29 15:07:43,152 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] checkpoint have error, cancel the pipeline
2023-05-29 15:07:43,153 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,156 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,157 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,158 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,160 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] operator to member [bigdata07]:5801
2023-05-29 15:07:43,160 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] operator to member [bigdata07]:5801
2023-05-29 15:07:43,160 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] operator to member [bigdata07]:5801
2023-05-29 15:07:43,161 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn to end state CANCELED.
2023-05-29 15:07:43,161 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (2/2)] end with state CANCELED
2023-05-29 15:07:43,161 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn to end state CANCELED.
2023-05-29 15:07:43,161 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] end with state CANCELED
2023-05-29 15:07:43,162 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn to end state CANCELED.
2023-05-29 15:07:43,162 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-default]-SourceTask (1/2)] end with state CANCELED
2023-05-29 15:07:43,193 INFO org.apache.seatunnel.engine.server.master.JobMaster - release the pipeline Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] resource
2023-05-29 15:07:43,197 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] end with state FAILED
2023-05-29 15:07:43,197 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - cancel job Job cus_basic_applet (715450079284035585) because makeJobEndWhenPipelineEnded is true
2023-05-29 15:07:43,199 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job cus_basic_applet (715450079284035585) turn from state RUNNING to CANCELLING.
2023-05-29 15:07:43,200 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)] is in end state FAILED, can not be cancel
2023-05-29 15:07:43,201 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job cus_basic_applet (715450079284035585) turn from state CANCELLING to FAILING.
2023-05-29 15:07:43,202 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job cus_basic_applet (715450079284035585) turn from state FAILING to FAILED.
2023-05-29 15:07:43,224 INFO com.hazelcast.internal.server.tcp.TcpServerConnection - [bigdata05]:5801 [seatunnel_mycluster] [5.1] Connection[id=6, /10.6.1.16:5801->/10.6.1.16:45233, qualifier=null, endpoint=[10.6.1.16]:45233, remoteUuid=9b983f15-396a-4d6e-a9bf-2fb02eb9db82, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org