You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Alex Soto <al...@envieta.com> on 2018/04/23 18:48:06 UTC
akka.remote.OversizedPayloadException
Hello,
I am using Flink version 1.4.2. When I try to run my application on a Yarn cluster, I get this error:
2018-04-23 14:08:13,346 ERROR akka.remote.EndpointWriter - Transient association error (association remains live)
akka.remote.OversizedPayloadException: Discarding oversized payload sent to Actor[akka.tcp://flink@XXXX:37877/user/jobmanager#1227317945]: max allowed size 10485760 bytes, actual size of encoded class org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage was 213691138 bytes.
I am trying to override the default limit in my flink-conf.yaml file:
env.java.opts: -Dakka.framesize=314572800b -Dakka.client.timeout=10min
I can verify this setting shows up in the Job Manager Web UI:
Key Value
env.java.opts -Dakka.framesize=314572800b -Dakka.client.timeout=10min
I am submitting the Job with the command line client:
flink run -m …
No matter what I do, I always get the same error message with the same default limit. Please help, I am not finding any information on how to override the default limit.
Best regards,
Alex soto
Re: akka.remote.OversizedPayloadException
Posted by Ted Yu <yu...@gmail.com>.
I noticed trailing 'b' in your setting.
Have you tried without the trailing 'b' ?
Cheers
On Mon, Apr 23, 2018 at 11:48 AM, Alex Soto <al...@envieta.com> wrote:
> Hello,
>
> I am using Flink version 1.4.2. When I try to run my application on a
> Yarn cluster, I get this error:
>
> 2018-04-23 14:08:13,346 ERROR akka.remote.EndpointWriter
> - Transient association error (association remains live)
> akka.remote.OversizedPayloadException: Discarding oversized payload sent
> to Actor[akka.tcp://flink@XXXX:37877/user/jobmanager#1227317945]: max
> allowed size 10485760 bytes, actual size of encoded
> class org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage
> was 213691138 bytes.
>
>
> I am trying to override the default limit in my *flink-conf.yaml *file:
>
> env.java.opts: -Dakka.framesize=314572800b -Dakka.client.timeout=10min
>
>
> I can verify this setting shows up in the Job Manager Web UI:
>
> Key Value
> env.java.opts -Dakka.framesize=314572800b -Dakka.client.timeout=10min
>
>
> I am submitting the Job with the command line client:
>
> flink run -m …
>
> No matter what I do, I always get the same error message with the same
> default limit. Please help, I am not finding any information on how to
> override the default limit.
>
> Best regards,
> Alex soto
>
>
>
Re: akka.remote.OversizedPayloadException
Posted by Gary Yao <ga...@data-artisans.com>.
Hi Alex,
Can you try add the following two lines to your flink-conf.yaml?
akka.framesize: 314572800b
akka.client.timeout: 10min
AFAIK it is not needed to use Java system properties here.
Best,
Gary
On Mon, Apr 23, 2018 at 8:48 PM, Alex Soto <al...@envieta.com> wrote:
> Hello,
>
> I am using Flink version 1.4.2. When I try to run my application on a
> Yarn cluster, I get this error:
>
> 2018-04-23 14:08:13,346 ERROR akka.remote.EndpointWriter
> - Transient association error (association remains live)
> akka.remote.OversizedPayloadException: Discarding oversized payload sent
> to Actor[akka.tcp://flink@XXXX:37877/user/jobmanager#1227317945]: max
> allowed size 10485760 bytes, actual size of encoded
> class org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage
> was 213691138 bytes.
>
>
> I am trying to override the default limit in my *flink-conf.yaml *file:
>
> env.java.opts: -Dakka.framesize=314572800b -Dakka.client.timeout=10min
>
>
> I can verify this setting shows up in the Job Manager Web UI:
>
> Key Value
> env.java.opts -Dakka.framesize=314572800b -Dakka.client.timeout=10min
>
>
> I am submitting the Job with the command line client:
>
> flink run -m …
>
> No matter what I do, I always get the same error message with the same
> default limit. Please help, I am not finding any information on how to
> override the default limit.
>
> Best regards,
> Alex soto
>
>
>