You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by simple <10...@qq.com> on 2023/02/24 08:17:00 UTC
我使用 start-cluster.sh 运行的flink，我的jar运行一段时间后被关闭了，这个时间是不固定的
2023-02-24 16:05:26,104 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source (1/1) (9644f5bc58ed204c6d31bb133156fa95_bc764cd8ddf7a0cff126f51c16239658_0_0) switched from RUNNING to CANCELING.
2023-02-24 16:05:26,108 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Flat Map (1/2) (9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_0_0) switched from RUNNING to CANCELING.
2023-02-24 16:05:26,108 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Flat Map (2/2) (9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_1_0) switched from RUNNING to CANCELING.
2023-02-24 16:05:26,108 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - KeyedProcess -> Sink: Unnamed (2/2) (9644f5bc58ed204c6d31bb133156fa95_e70bbd798b564e0a50e10e343f1ac56b_1_0) switched from RUNNING to CANCELING.
2023-02-24 16:05:26,137 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Flat Map (1/2) (9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_0_0) switched from CANCELING to CANCELED.
2023-02-24 16:05:26,140 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Flat Map (2/2) (9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_1_0) switched from CANCELING to CANCELED.
2023-02-24 16:05:26,141 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - KeyedProcess -> Sink: Unnamed (2/2) (9644f5bc58ed204c6d31bb133156fa95_e70bbd798b564e0a50e10e343f1ac56b_1_0) switched from CANCELING to CANCELED.
2023-02-24 16:05:26,145 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 299ff4a6ea193d7c110a19a3de33edab: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2023-02-24 16:05:26,152 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source (1/1) (9644f5bc58ed204c6d31bb133156fa95_bc764cd8ddf7a0cff126f51c16239658_0_0) switched from CANCELING to CANCELED.
2023-02-24 16:05:26,155 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 299ff4a6ea193d7c110a19a3de33edab
2023-02-24 16:05:26,154 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job [10204-08]BatchRunningTimesLBSHBaseJobApp (299ff4a6ea193d7c110a19a3de33edab) switched from state FAILING to FAILED.
org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:139) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:83) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:256) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:247) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:240) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:738) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:715) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:78) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:477) ~[flink-runtime-1.16.0.jar:1.16.0]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:309) ~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:307) ~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:222) ~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84) ~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168) ~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) [?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) [?:?]
Caused by: org.apache.flink.runtime.taskmanager.AsynchronousException: Caught exception while processing timer.
at org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1568) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1543) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1683) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) ~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: org.apache.flink.streaming.runtime.tasks.TimerException: org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException: Could not forward element to next operator
at org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1683) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) ~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException: Could not forward element to next operator
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:99) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.transformation(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:87) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.resultOut(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:76) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimerCallback(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:48) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimer(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:43) ~[?:?]
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.invokeUserFunction(KeyedProcessOperator.java:91) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.onProcessingTime(KeyedProcessOperator.java:76) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.onProcessingTime(InternalTimerServiceImpl.java:284) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1681) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) ~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: java.lang.RuntimeException: exception
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.checkErrorAndRethrow(BatchRunningTimesHBaseUdfRichSinkFunction.java:264) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:257) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:235) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:37) ~[?:?]
at org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:54) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.transformation(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:87) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.resultOut(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:76) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimerCallback(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:48) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimer(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:43) ~[?:?]
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.invokeUserFunction(KeyedProcessOperator.java:91) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.onProcessingTime(KeyedProcessOperator.java:76) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.onProcessingTime(InternalTimerServiceImpl.java:284) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1681) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) ~[flink-streaming-java-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) ~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) ~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043) ~[?:?]
at java.util.ArrayList$Itr.next(ArrayList.java:997) ~[?:?]
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:189) ~[hbase-client-2.2.2.jar:2.2.2]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:244) ~[?:?]
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.lambda$open$0(BatchRunningTimesHBaseUdfRichSinkFunction.java:97) ~[?:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-02-24 16:05:26,158 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Stopping checkpoint coordinator for job 299ff4a6ea193d7c110a19a3de33edab.
2023-02-24 16:05:26,178 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Job 299ff4a6ea193d7c110a19a3de33edab reached terminal state FAILED.
org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:139)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:83)
at org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:256)
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:247)
at org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:240)
at org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:738)
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:715)
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:78)
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:477)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:309)
at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:307)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:222)
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:537)
at akka.actor.Actor.aroundReceive$(Actor.scala:535)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)
at akka.actor.ActorCell.invoke(ActorCell.scala:548)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: org.apache.flink.runtime.taskmanager.AsynchronousException: Caught exception while processing timer.
at org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1568)
at org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1543)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1683)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: TimerException{org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException: Could not forward element to next operator}
... 15 more
Caused by: org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException: Could not forward element to next operator
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:99)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)
at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)
at org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.transformation(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:87)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.resultOut(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:76)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimerCallback(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:48)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimer(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:43)
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.invokeUserFunction(KeyedProcessOperator.java:91)
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.onProcessingTime(KeyedProcessOperator.java:76)
at org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.onProcessingTime(InternalTimerServiceImpl.java:284)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1681)
... 14 more
Caused by: java.lang.RuntimeException: exception
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.checkErrorAndRethrow(BatchRunningTimesHBaseUdfRichSinkFunction.java:264)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:257)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:235)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:37)
at org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:54)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82)
... 27 more
Caused by: java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:189)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:244)
at com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.lambda$open$0(BatchRunningTimesHBaseUdfRichSinkFunction.java:97)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 more
2023-02-24 16:05:26,225 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Job 299ff4a6ea193d7c110a19a3de33edab has been registered for cleanup in the JobResultStore after reaching a terminal state.
2023-02-24 16:05:26,231 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Stopping the JobMaster for job '[10204-08]BatchRunningTimesLBSHBaseJobApp' (299ff4a6ea193d7c110a19a3de33edab).
2023-02-24 16:05:26,237 INFO org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore [] - Shutting down
2023-02-24 16:05:26,239 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Disconnect TaskExecutor localhost:45013-1b5bd3 because: Stopping JobMaster for job '[10204-08]BatchRunningTimesLBSHBaseJobApp' (299ff4a6ea193d7c110a19a3de33edab).
2023-02-24 16:05:26,240 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [ce80357b0859df0b268566294786c8a9].
2023-02-24 16:05:26,242 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [7f78bde0f41f2cf0931a01a54db98bfb].
2023-02-24 16:05:26,243 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Close ResourceManager connection 4d0adaf4e63930c3693def906b4a3e7e: Stopping JobMaster for job '[10204-08]BatchRunningTimesLBSHBaseJobApp' (299ff4a6ea193d7c110a19a3de33edab).
2023-02-24 16:05:26,246 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Disconnect job manager 00000000000000000000000000000000@akka.tcp://flink@localhost:6123/user/rpc/jobmanager_2 for job 299ff4a6ea193d7c110a19a3de33edab from the resource manager.
冉庆超
1028108734@qq.com