You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Till Rohrmann (Jira)" <ji...@apache.org> on 2020/05/18 09:24:00 UTC

[jira] [Commented] (FLINK-17750) YARNHighAvailabilityITCase.testKillYarnSessionClusterEntrypoint failed on azure

    [ https://issues.apache.org/jira/browse/FLINK-17750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110074#comment-17110074 ] 

Till Rohrmann commented on FLINK-17750:
---------------------------------------

The problem might be that a previous Yarn test did not properly stop:

{code}
 Suppressed: java.lang.AssertionError: There is at least one application on the cluster that is not finished.[App application_1 589586092047_0002 is in state RUNNING.]
 2020-05-15T23:42:29.5363571Z            at org.junit.Assert.fail(Assert.java:88)
 2020-05-15T23:42:29.5364060Z            at org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:286)
 2020-05-15T23:42:29.5364448Z            at org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:259)
 2020-05-15T23:42:29.5364745Z            ... 36 more
{code}

> YARNHighAvailabilityITCase.testKillYarnSessionClusterEntrypoint failed on azure
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-17750
>                 URL: https://issues.apache.org/jira/browse/FLINK-17750
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination
>    Affects Versions: 1.11.0
>            Reporter: Roman Khachatryan
>            Priority: Critical
>              Labels: test-stability
>
> [https://dev.azure.com/khachatryanroman/810e80cc-0656-4d3c-9d8c-186764456a01/_apis/build/builds/6/logs/156]
>  
> {code:java}
>  2020-05-15T23:42:29.5307581Z [ERROR] testKillYarnSessionClusterEntrypoint(org.apache.flink.yarn.YARNHighAvailabilityITCase)  Time elapsed: 21.68 s  <<< ERROR!
>  2020-05-15T23:42:29.5308406Z java.util.concurrent.ExecutionException:
>  2020-05-15T23:42:29.5308864Z org.apache.flink.runtime.rest.util.RestClientException: [Internal server error., <Exception on server side:
>  2020-05-15T23:42:29.5309678Z java.util.concurrent.TimeoutException: Invocation of public abstract java.util.concurrent.CompletableFuture org.apache.flink.runt ime.dispatcher.DispatcherGateway.requestJob(org.apache.flink.api.common.JobID,org.apache.flink.api.common.time.Time) timed out.
>  2020-05-15T23:42:29.5310322Z    at com.sun.proxy.$Proxy33.requestJob(Unknown Source)
>  2020-05-15T23:42:29.5311018Z    at org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCach e.java:103)
>  2020-05-15T23:42:29.5311704Z    at org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraph(DefaultExecutionGraphCache.java:7 1)
>  2020-05-15T23:42:29.5312355Z    at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:75 )
>  2020-05-15T23:42:29.5312924Z    at org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:73)
>  2020-05-15T23:42:29.5313423Z    at org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:172)
>  2020-05-15T23:42:29.5314497Z    at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:81)
>  2020-05-15T23:42:29.5315083Z    at java.util.Optional.ifPresent(Optional.java:159)
>  2020-05-15T23:42:29.5315474Z    at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:46)
>  2020-05-15T23:42:29.5315979Z    at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:78)
>  2020-05-15T23:42:29.5316520Z    at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  2020-05-15T23:42:29.5317092Z    at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:10 5)
>  2020-05-15T23:42:29.5317705Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5318586Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5319249Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5319729Z    at org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:110)
>  2020-05-15T23:42:29.5320136Z    at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:89)
>  2020-05-15T23:42:29.5320742Z    at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:54)
>  2020-05-15T23:42:29.5321195Z    at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:10 5)
>  2020-05-15T23:42:29.5321730Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5322263Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5322806Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5323335Z    at org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
>  2020-05-15T23:42:29.5323849Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5324390Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5324910Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5325920Z    at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:174)
>  2020-05-15T23:42:29.5326353Z    at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:68)
>  2020-05-15T23:42:29.5326867Z    at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:10 5)
>  2020-05-15T23:42:29.5327532Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5328140Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5328748Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5329391Z    at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRea d(CombinedChannelDuplexHandler.java:438)
>  2020-05-15T23:42:29.5330028Z    at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328)
>  2020-05-15T23:42:29.5330593Z    at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:302)
>  2020-05-15T23:42:29.5331149Z    at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java: 253)
>  2020-05-15T23:42:29.5331750Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5332346Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5332956Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5333552Z    at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java: 1421)
>  2020-05-15T23:42:29.5334135Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5334748Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5335383Z    at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
>  2020-05-15T23:42:29.5335952Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:1 63)
>  2020-05-15T23:42:29.5336485Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
>  2020-05-15T23:42:29.5337019Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
>  2020-05-15T23:42:29.5337552Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
>  2020-05-15T23:42:29.5338023Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
>  2020-05-15T23:42:29.5338689Z    at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
>  2020-05-15T23:42:29.5339358Z    at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  2020-05-15T23:42:29.5339869Z    at java.lang.Thread.run(Thread.java:748)
>  2020-05-15T23:42:29.5341122Z Caused by: akka.pattern.AskTimeoutException: Recipient [Actor[akka://flink/user/rpc/dispatcher_1#-1875884516]] had already been t erminated. Message of type [org.apache.flink.runtime.rpc.messages.LocalFencedMessage].
>  2020-05-15T23:42:29.5341660Z    at akka.pattern.AskableActorRef$.recipientTerminatedException(AskSupport.scala:288)
>  2020-05-15T23:42:29.5342018Z    at akka.pattern.AskableActorRef$.internalAsk$extension(AskSupport.scala:330)
>  2020-05-15T23:42:29.5342321Z    at akka.pattern.AskSupport$class.ask(AskSupport.scala:81)
>  2020-05-15T23:42:29.5342592Z    at akka.pattern.package$.ask(package.scala:42)
>  2020-05-15T23:42:29.5342833Z    at akka.pattern.Patterns$.ask(Patterns.scala:160)
>  2020-05-15T23:42:29.5343180Z    at akka.pattern.Patterns.ask(Patterns.scala)
>  2020-05-15T23:42:29.5343485Z    at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.ask(AkkaInvocationHandler.java:361)
>  2020-05-15T23:42:29.5343911Z    at org.apache.flink.runtime.rpc.akka.FencedAkkaInvocationHandler.ask(FencedAkkaInvocationHandler.java:123)
>  2020-05-15T23:42:29.5344323Z    at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.invokeRpc(AkkaInvocationHandler.java:222)
>  2020-05-15T23:42:29.5344737Z    at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.invoke(AkkaInvocationHandler.java:134)
>  2020-05-15T23:42:29.5345147Z    at org.apache.flink.runtime.rpc.akka.FencedAkkaInvocationHandler.invoke(FencedAkkaInvocationHandler.java:79)
>  2020-05-15T23:42:29.5345443Z    ... 51 more
>  2020-05-15T23:42:29.5345533Z
>  2020-05-15T23:42:29.5345682Z End of exception on server side>]
>  2020-05-15T23:42:29.5345940Z    at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
>  2020-05-15T23:42:29.5346289Z    at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
>  2020-05-15T23:42:29.5346680Z    at org.apache.flink.yarn.YARNHighAvailabilityITCase.lambda$waitUntilJobIsRunning$4(YARNHighAvailabilityITCase.java:312)
>  2020-05-15T23:42:29.5347126Z    at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:123)
>  2020-05-15T23:42:29.5347520Z    at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:119)
>  2020-05-15T23:42:29.5347950Z    at org.apache.flink.yarn.YARNHighAvailabilityITCase.waitUntilJobIsRunning(YARNHighAvailabilityITCase.java:310)
>  2020-05-15T23:42:29.5348430Z    at org.apache.flink.yarn.YARNHighAvailabilityITCase.lambda$testKillYarnSessionClusterEntrypoint$0(YARNHighAvailabilityITCase.j*ava:168)
>  2020-05-15T23:42:29.5348835Z    at org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:258)
>  2020-05-15T23:42:29.5349420Z    at org.apache.flink.yarn.YARNHighAvailabilityITCase.testKillYarnSessionClusterEntrypoint(YARNHighAvailabilityITCase.java:156)
>  2020-05-15T23:42:29.5349984Z    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  2020-05-15T23:42:29.5350385Z    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  2020-05-15T23:42:29.5350772Z    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  2020-05-15T23:42:29.5351125Z    at java.lang.reflect.Method.invoke(Method.java:498)
>  2020-05-15T23:42:29.5351459Z    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>  2020-05-15T23:42:29.5351868Z    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  2020-05-15T23:42:29.5352274Z    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>  2020-05-15T23:42:29.5352661Z    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>  2020-05-15T23:42:29.5353219Z    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>  2020-05-15T23:42:29.5353559Z    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>  2020-05-15T23:42:29.5353886Z    at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>  2020-05-15T23:42:29.5354163Z    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>  2020-05-15T23:42:29.5354454Z    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>  2020-05-15T23:42:29.5354776Z    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>  2020-05-15T23:42:29.5355144Z    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>  2020-05-15T23:42:29.5355459Z    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>  2020-05-15T23:42:29.5355764Z    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>  2020-05-15T23:42:29.5356062Z    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>  2020-05-15T23:42:29.5356382Z    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>  2020-05-15T23:42:29.5356681Z    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>  2020-05-15T23:42:29.5357068Z    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>  2020-05-15T23:42:29.5357432Z    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>  2020-05-15T23:42:29.5357757Z    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
>  2020-05-15T23:42:29.5358088Z    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
>  2020-05-15T23:42:29.5358402Z    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
>  2020-05-15T23:42:29.5358705Z    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>  2020-05-15T23:42:29.5358976Z    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>  2020-05-15T23:42:29.5359307Z    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
>  2020-05-15T23:42:29.5359674Z    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
>  2020-05-15T23:42:29.5360252Z    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
>  2020-05-15T23:42:29.5360862Z    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
>  2020-05-15T23:42:29.5361296Z    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
>  2020-05-15T23:42:29.5361723Z    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
>  2020-05-15T23:42:29.5362125Z    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
>  2020-05-15T23:42:29.5362503Z    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
>  2020-05-15T23:42:29.5362969Z    Suppressed: java.lang.AssertionError: There is at least one application on the cluster that is not finished.[App application_1 589586092047_0002 is in state RUNNING.]
>  2020-05-15T23:42:29.5363571Z            at org.junit.Assert.fail(Assert.java:88)
>  2020-05-15T23:42:29.5364060Z            at org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:286)
>  2020-05-15T23:42:29.5364448Z            at org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:259)
>  2020-05-15T23:42:29.5364745Z            ... 36 more
>  2020-05-15T23:42:29.5365232Z Caused by: org.apache.flink.runtime.rest.util.RestClientException: [Internal server error., <Exception on server side:
>  2020-05-15T23:42:29.5365862Z java.util.concurrent.TimeoutException: Invocation of public abstract java.util.concurrent.CompletableFuture org.apache.flink.runt ime.dispatcher.DispatcherGateway.requestJob(org.apache.flink.api.common.JobID,org.apache.flink.api.common.time.Time) timed out.
>  2020-05-15T23:42:29.5366535Z    at com.sun.proxy.$Proxy33.requestJob(Unknown Source)
>  2020-05-15T23:42:29.5366919Z    at org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCach e.java:103)
>  2020-05-15T23:42:29.5367421Z    at org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraph(DefaultExecutionGraphCache.java:7 1)
>  2020-05-15T23:42:29.5367925Z    at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:75 )
>  2020-05-15T23:42:29.5368376Z    at org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:73)
>  2020-05-15T23:42:29.5368902Z    at org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:172)
>  2020-05-15T23:42:29.5369342Z    at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:81)
>  2020-05-15T23:42:29.5369695Z    at java.util.Optional.ifPresent(Optional.java:159)
>  2020-05-15T23:42:29.5370003Z    at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:46)
>  2020-05-15T23:42:29.5370430Z    at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:78)
>  2020-05-15T23:42:29.5370872Z    at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  2020-05-15T23:42:29.5371391Z    at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:10 5)
>  2020-05-15T23:42:29.5371922Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5372438Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
>  2020-05-15T23:42:29.5372963Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5373420Z    at org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:110)
>  2020-05-15T23:42:29.5373804Z    at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:89)
>  2020-05-15T23:42:29.5374211Z    at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:54)
>  2020-05-15T23:42:29.5374655Z    at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:10 5)
>  2020-05-15T23:42:29.5375352Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5376349Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5376957Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5377544Z    at org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
>  2020-05-15T23:42:29.5378119Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5378804Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5379469Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5379997Z    at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:174)
>  2020-05-15T23:42:29.5381976Z    at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:68)
>  2020-05-15T23:42:29.5382435Z    at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:10 5)
>  2020-05-15T23:42:29.5382951Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5383467Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5384003Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5384561Z    at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRea d(CombinedChannelDuplexHandler.java:438)
>  2020-05-15T23:42:29.5385116Z    at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328)
>  2020-05-15T23:42:29.5385601Z    at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:302)
>  2020-05-15T23:42:29.5386075Z    at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java: 253)
>  2020-05-15T23:42:29.5386588Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5387217Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5387765Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext .java:352)
>  2020-05-15T23:42:29.5388263Z    at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java: 1421)
>  2020-05-15T23:42:29.5388785Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:374)
>  2020-05-15T23:42:29.5389313Z    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerConte xt.java:360)
>  2020-05-15T23:42:29.5389799Z    at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
>  2020-05-15T23:42:29.5390295Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:1 63)
>  2020-05-15T23:42:29.5390755Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
>  2020-05-15T23:42:29.5391213Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
>  2020-05-15T23:42:29.5391659Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
>  2020-05-15T23:42:29.5392081Z    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
>  2020-05-15T23:42:29.5392527Z    at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
>  2020-05-15T23:42:29.5392978Z    at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  2020-05-15T23:42:29.5393314Z    at java.lang.Thread.run(Thread.java:748)
>  2020-05-15T23:42:29.5394130Z Caused by: akka.pattern.AskTimeoutException: Recipient [Actor[akka://flink/user/rpc/dispatcher_1#-1875884516]] had already been t erminated. Message of type [org.apache.flink.runtime.rpc.messages.LocalFencedMessage].
>  2020-05-15T23:42:29.5394945Z    at akka.pattern.AskableActorRef$.recipientTerminatedException(AskSupport.scala:288)
>  2020-05-15T23:42:29.5395309Z    at akka.pattern.AskableActorRef$.internalAsk$extension(AskSupport.scala:330)
>  2020-05-15T23:42:29.5395831Z    at akka.pattern.AskSupport$class.ask(AskSupport.scala:81)
>  2020-05-15T23:42:29.5396125Z    at akka.pattern.package$.ask(package.scala:42)
>  2020-05-15T23:42:29.5396388Z    at akka.pattern.Patterns$.ask(Patterns.scala:160)
>  2020-05-15T23:42:29.5396994Z    at akka.pattern.Patterns.ask(Patterns.scala)
>  2020-05-15T23:42:29.5397333Z    at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.ask(AkkaInvocationHandler.java:361)
>  2020-05-15T23:42:29.5397823Z    at org.apache.flink.runtime.rpc.akka.FencedAkkaInvocationHandler.ask(FencedAkkaInvocationHandler.java:123)
>  2020-05-15T23:42:29.5398457Z    at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.invokeRpc(AkkaInvocationHandler.java:222)
>  2020-05-15T23:42:29.5398923Z    at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.invoke(AkkaInvocationHandler.java:134)
>  2020-05-15T23:42:29.5399383Z    at org.apache.flink.runtime.rpc.akka.FencedAkkaInvocationHandler.invoke(FencedAkkaInvocationHandler.java:79)
>  2020-05-15T23:42:29.5399708Z    ... 51 more
>  2020-05-15T23:42:29.5399974Z
>  2020-05-15T23:42:29.5400132Z End of exception on server side>]
>  2020-05-15T23:42:29.5400567Z    at org.apache.flink.runtime.rest.RestClient.parseResponse(RestClient.java:390)
>  2020-05-15T23:42:29.5400930Z    at org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$3(RestClient.java:374)
>  2020-05-15T23:42:29.5401283Z    at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:966)
>  2020-05-15T23:42:29.5401652Z    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
>  2020-05-15T23:42:29.5402067Z    at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
>  2020-05-15T23:42:29.5402452Z    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  2020-05-15T23:42:29.5402800Z    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  2020-05-15T23:42:29.5403098Z    at java.lang.Thread.run(Thread.java:748)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)