You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by 崔深圳 <cu...@126.com> on 2022/03/07 11:46:35 UTC
k8s native session 问题咨询
k8s native session 模式下，配置了ha，job_manager 的数量为3，然后web ui，通过rest service访问，总是路由到非master节点，有什么办法使其稳定吗？
Re:Re: k8s native session 问题咨询
Posted by 崔深圳 <cu...@126.com>.
web ui报错：请求这个接口： /jobs/overview，时而报错， <Exception on server side:\norg.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException: Failed to serialize the result for RPC call : requestMultipleJobDetails.\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:417)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$2(AkkaRpcActor.java:373)\n\tat java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:848)\n\tat java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2168)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendAsyncResponse(AkkaRpcActor.java:365)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:332)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)\n\tat org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)\n\tat akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)\n\tat akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)\n\tat scala.PartialFunction.applyOrElse(PartialFunction.scala:123)\n\tat scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)\n\tat akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)\n\tat scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)\n\tat scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat akka.actor.Actor.aroundReceive(Actor.scala:537)\n\tat akka.actor.Actor.aroundReceive$(Actor.scala:535)\n\tat akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)\n\tat akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)\n\tat akka.actor.ActorCell.invoke(ActorCell.scala:548)\n\tat akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)\n\tat akka.dispatch.Mailbox.run(Mailbox.scala:231)\n\tat akka.dispatch.Mailbox.exec(Mailbox.scala:243)\n\tat java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)\n\tat java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)\n\tat java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)\n\tat java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)\nCaused by: java.io.NotSerializableException: java.util.HashMap$Values\n\tat java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)\n\tat java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)\n\tat java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)\n\tat java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)\n\tat java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)\n\tat java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)\n\tat org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcSerializedValue.valueOf(AkkaRpcSerializedValue.java:66)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:400)\n\t... 29 more\n\nEnd of exception on server side>"
在 2022-03-09 09:56:21，"yu'an huang" <h....@gmail.com> 写道：
>你好，路由到非master节点会有什么问题吗，非master节点在处理请求时应该会通过HA服务找到Active的Job manager, 然后向Active Job manager拿到结果再返回给client.
>
>> On 7 Mar 2022, at 7:46 PM, 崔深圳 <cu...@126.com> wrote:
>>
>> k8s native session 模式下，配置了ha，job_manager 的数量为3，然后web ui，通过rest service访问，总是路由到非master节点，有什么办法使其稳定吗？
>
Re: k8s native session 问题咨询
Posted by Yang Wang <da...@gmail.com>.
多谢yu'an huang的补充
yu'an huang <h....@gmail.com> 于2022年3月9日周三 16:21写道：
> Hi Yang,
>
> 我看到你发的issue中是跟Resource ProfileInfo不可被序列化有关，查了下感觉应该是在这两个issue中被修复的：
> https://issues.apache.org/jira/browse/FLINK-25732 <
> https://issues.apache.org/jira/browse/FLINK-25732>，
> https://issues.apache.org/jira/browse/FLINK-25837 <
> https://issues.apache.org/jira/browse/FLINK-25837>。
>
> 相关的PR是https://github.com/apache/flink/pull/18422 <
> https://github.com/apache/flink/pull/18422>
>
> 按照Issue所说应该换成
> 1.15.0 <
> https://issues.apache.org/jira/issues/?jql=project+%3D+FLINK+AND+fixVersion+%3D+1.15.0>,
> 1.13.6 <
> https://issues.apache.org/jira/issues/?jql=project+%3D+FLINK+AND+fixVersion+%3D+1.13.6>,
> 1.14.4 <
> https://issues.apache.org/jira/issues/?jql=project+%3D+FLINK+AND+fixVersion+%3D+1.14.4
> >以上的版本应该就没问题了。
>
>
>
> > On 9 Mar 2022, at 12:00 PM, Yang Wang <da...@gmail.com> wrote:
> >
> > 你用新版本试一下，看着是已经修复了
> >
> > https://issues.apache.org/jira/browse/FLINK-19212
> >
> > Best,
> > Yang
> >
> > 崔深圳 <cu...@126.com> 于2022年3月9日周三 10:31写道：
> >
> >>
> >>
> >>
> >> web ui报错：请求这个接口： /jobs/overview，时而报错， <Exception on server
> >> side:\norg.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
> >> Failed to serialize the result for RPC call :
> >> requestMultipleJobDetails.\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:417)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$2(AkkaRpcActor.java:373)\n\tat
> >>
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat
> >>
> java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:848)\n\tat
> >>
> java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2168)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendAsyncResponse(AkkaRpcActor.java:365)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:332)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)\n\tat
> >> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)\n\tat
> >> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)\n\tat
> >> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)\n\tat
> >> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)\n\tat
> >> akka.japi.pf
> .UnitCaseStatement.applyOrElse(CaseStatements.scala:20)\n\tat
> >>
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)\n\tat
> >>
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
> >>
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
> >> akka.actor.Actor.aroundReceive(Actor.scala:537)\n\tat
> >> akka.actor.Actor.aroundReceive$(Actor.scala:535)\n\tat
> >> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)\n\tat
> >> akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)\n\tat
> >> akka.actor.ActorCell.invoke(ActorCell.scala:548)\n\tat
> >> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)\n\tat
> >> akka.dispatch.Mailbox.run(Mailbox.scala:231)\n\tat
> >> akka.dispatch.Mailbox.exec(Mailbox.scala:243)\n\tat
> >> java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)\n\tat
> >>
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)\n\tat
> >>
> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)\n\tat
> >>
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)\nCaused
> >> by: java.io.NotSerializableException: java.util.HashMap$Values\n\tat
> >>
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)\n\tat
> >>
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)\n\tat
> >>
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)\n\tat
> >>
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)\n\tat
> >>
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)\n\tat
> >>
> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)\n\tat
> >>
> org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcSerializedValue.valueOf(AkkaRpcSerializedValue.java:66)\n\tat
> >>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:400)\n\t...
> >> 29 more\n\nEnd of exception on server side>"
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> 在 2022-03-09 09:56:21，"yu'an huang" <h....@gmail.com> 写道：
> >>> 你好，路由到非master节点会有什么问题吗，非master节点在处理请求时应该会通过HA服务找到Active的Job manager,
> >> 然后向Active Job manager拿到结果再返回给client.
> >>>
> >>>> On 7 Mar 2022, at 7:46 PM, 崔深圳 <cu...@126.com> wrote:
> >>>>
> >>>> k8s native session 模式下，配置了ha，job_manager 的数量为3，然后web ui，通过rest
> >> service访问，总是路由到非master节点，有什么办法使其稳定吗？
> >>>
> >>
>
>
Re: k8s native session 问题咨询
Posted by yu'an huang <h....@gmail.com>.
Hi Yang,
我看到你发的issue中是跟Resource ProfileInfo不可被序列化有关，查了下感觉应该是在这两个issue中被修复的：
https://issues.apache.org/jira/browse/FLINK-25732 <https://issues.apache.org/jira/browse/FLINK-25732>，
https://issues.apache.org/jira/browse/FLINK-25837 <https://issues.apache.org/jira/browse/FLINK-25837>。
相关的PR是https://github.com/apache/flink/pull/18422 <https://github.com/apache/flink/pull/18422>
按照Issue所说应该换成
1.15.0 <https://issues.apache.org/jira/issues/?jql=project+%3D+FLINK+AND+fixVersion+%3D+1.15.0>, 1.13.6 <https://issues.apache.org/jira/issues/?jql=project+%3D+FLINK+AND+fixVersion+%3D+1.13.6>, 1.14.4 <https://issues.apache.org/jira/issues/?jql=project+%3D+FLINK+AND+fixVersion+%3D+1.14.4>以上的版本应该就没问题了。
> On 9 Mar 2022, at 12:00 PM, Yang Wang <da...@gmail.com> wrote:
>
> 你用新版本试一下，看着是已经修复了
>
> https://issues.apache.org/jira/browse/FLINK-19212
>
> Best,
> Yang
>
> 崔深圳 <cu...@126.com> 于2022年3月9日周三 10:31写道：
>
>>
>>
>>
>> web ui报错：请求这个接口： /jobs/overview，时而报错， <Exception on server
>> side:\norg.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
>> Failed to serialize the result for RPC call :
>> requestMultipleJobDetails.\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:417)\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$2(AkkaRpcActor.java:373)\n\tat
>> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat
>> java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:848)\n\tat
>> java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2168)\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendAsyncResponse(AkkaRpcActor.java:365)\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:332)\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)\n\tat
>> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78)\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)\n\tat
>> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)\n\tat
>> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)\n\tat
>> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)\n\tat
>> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)\n\tat
>> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)\n\tat
>> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)\n\tat
>> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
>> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
>> akka.actor.Actor.aroundReceive(Actor.scala:537)\n\tat
>> akka.actor.Actor.aroundReceive$(Actor.scala:535)\n\tat
>> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)\n\tat
>> akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)\n\tat
>> akka.actor.ActorCell.invoke(ActorCell.scala:548)\n\tat
>> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)\n\tat
>> akka.dispatch.Mailbox.run(Mailbox.scala:231)\n\tat
>> akka.dispatch.Mailbox.exec(Mailbox.scala:243)\n\tat
>> java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)\n\tat
>> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)\n\tat
>> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)\n\tat
>> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)\nCaused
>> by: java.io.NotSerializableException: java.util.HashMap$Values\n\tat
>> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)\n\tat
>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)\n\tat
>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)\n\tat
>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)\n\tat
>> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)\n\tat
>> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)\n\tat
>> org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcSerializedValue.valueOf(AkkaRpcSerializedValue.java:66)\n\tat
>> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:400)\n\t...
>> 29 more\n\nEnd of exception on server side>"
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> 在 2022-03-09 09:56:21，"yu'an huang" <h....@gmail.com> 写道：
>>> 你好，路由到非master节点会有什么问题吗，非master节点在处理请求时应该会通过HA服务找到Active的Job manager,
>> 然后向Active Job manager拿到结果再返回给client.
>>>
>>>> On 7 Mar 2022, at 7:46 PM, 崔深圳 <cu...@126.com> wrote:
>>>>
>>>> k8s native session 模式下，配置了ha，job_manager 的数量为3，然后web ui，通过rest
>> service访问，总是路由到非master节点，有什么办法使其稳定吗？
>>>
>>
Re: Re: k8s native session 问题咨询
Posted by Yang Wang <da...@gmail.com>.
你用新版本试一下，看着是已经修复了
https://issues.apache.org/jira/browse/FLINK-19212
Best,
Yang
崔深圳 <cu...@126.com> 于2022年3月9日周三 10:31写道：
>
>
>
> web ui报错：请求这个接口： /jobs/overview，时而报错， <Exception on server
> side:\norg.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
> Failed to serialize the result for RPC call :
> requestMultipleJobDetails.\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:417)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$2(AkkaRpcActor.java:373)\n\tat
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat
> java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:848)\n\tat
> java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2168)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendAsyncResponse(AkkaRpcActor.java:365)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:332)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)\n\tat
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)\n\tat
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)\n\tat
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)\n\tat
> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)\n\tat
> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)\n\tat
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)\n\tat
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)\n\tat
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
> akka.actor.Actor.aroundReceive(Actor.scala:537)\n\tat
> akka.actor.Actor.aroundReceive$(Actor.scala:535)\n\tat
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)\n\tat
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)\n\tat
> akka.actor.ActorCell.invoke(ActorCell.scala:548)\n\tat
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)\n\tat
> akka.dispatch.Mailbox.run(Mailbox.scala:231)\n\tat
> akka.dispatch.Mailbox.exec(Mailbox.scala:243)\n\tat
> java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)\n\tat
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)\n\tat
> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)\n\tat
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)\nCaused
> by: java.io.NotSerializableException: java.util.HashMap$Values\n\tat
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)\n\tat
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)\n\tat
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)\n\tat
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)\n\tat
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)\n\tat
> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)\n\tat
> org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcSerializedValue.valueOf(AkkaRpcSerializedValue.java:66)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:400)\n\t...
> 29 more\n\nEnd of exception on server side>"
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> 在 2022-03-09 09:56:21，"yu'an huang" <h....@gmail.com> 写道：
> >你好，路由到非master节点会有什么问题吗，非master节点在处理请求时应该会通过HA服务找到Active的Job manager,
> 然后向Active Job manager拿到结果再返回给client.
> >
> >> On 7 Mar 2022, at 7:46 PM, 崔深圳 <cu...@126.com> wrote:
> >>
> >> k8s native session 模式下，配置了ha，job_manager 的数量为3，然后web ui，通过rest
> service访问，总是路由到非master节点，有什么办法使其稳定吗？
> >
>
Re:Re: k8s native session 问题咨询
Posted by 崔深圳 <cu...@126.com>.
web ui报错：请求这个接口： /jobs/overview，时而报错， <Exception on server side:\norg.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException: Failed to serialize the result for RPC call : requestMultipleJobDetails.\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:417)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$2(AkkaRpcActor.java:373)\n\tat java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:848)\n\tat java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2168)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendAsyncResponse(AkkaRpcActor.java:365)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:332)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)\n\tat org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)\n\tat akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)\n\tat akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)\n\tat scala.PartialFunction.applyOrElse(PartialFunction.scala:123)\n\tat scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)\n\tat akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)\n\tat scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)\n\tat scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat akka.actor.Actor.aroundReceive(Actor.scala:537)\n\tat akka.actor.Actor.aroundReceive$(Actor.scala:535)\n\tat akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)\n\tat akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)\n\tat akka.actor.ActorCell.invoke(ActorCell.scala:548)\n\tat akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)\n\tat akka.dispatch.Mailbox.run(Mailbox.scala:231)\n\tat akka.dispatch.Mailbox.exec(Mailbox.scala:243)\n\tat java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)\n\tat java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)\n\tat java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)\n\tat java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)\nCaused by: java.io.NotSerializableException: java.util.HashMap$Values\n\tat java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)\n\tat java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)\n\tat java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)\n\tat java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)\n\tat java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)\n\tat java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)\n\tat org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcSerializedValue.valueOf(AkkaRpcSerializedValue.java:66)\n\tat org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:400)\n\t... 29 more\n\nEnd of exception on server side>"
在 2022-03-09 09:56:21，"yu'an huang" <h....@gmail.com> 写道：
>你好，路由到非master节点会有什么问题吗，非master节点在处理请求时应该会通过HA服务找到Active的Job manager, 然后向Active Job manager拿到结果再返回给client.
>
>> On 7 Mar 2022, at 7:46 PM, 崔深圳 <cu...@126.com> wrote:
>>
>> k8s native session 模式下，配置了ha，job_manager 的数量为3，然后web ui，通过rest service访问，总是路由到非master节点，有什么办法使其稳定吗？
>
Re: k8s native session 问题咨询
Posted by yu'an huang <h....@gmail.com>.
你好，路由到非master节点会有什么问题吗，非master节点在处理请求时应该会通过HA服务找到Active的Job manager, 然后向Active Job manager拿到结果再返回给client.
> On 7 Mar 2022, at 7:46 PM, 崔深圳 <cu...@126.com> wrote:
>
> k8s native session 模式下，配置了ha，job_manager 的数量为3，然后web ui，通过rest service访问，总是路由到非master节点，有什么办法使其稳定吗？