Exception during job submission in RemoteEnvironment

[Andreas Kunft <an...@tu-berlin.de>]

Hello,

I want to execute a Job from Java on an already running instance of 
Flink. I connect via the RemoteEnvironment. Everything goes fine, but 
then I get an exception from akka telling me the payload size is exceeded:

15-07-16 17:47:27 [ERROR] Transient association error (association 
remains live)
akka.remote.OversizedPayloadException: Discarding oversized payload sent 
to Actor[akka.tcp://flink@127.0.0.1:6123/user/jobmanager#697056301]: max 
allowed size 10485760 bytes, actual size of encoded class 
org.apache.flink.runtime.messages.JobManagerMessages$SubmitJob was 
28318303 bytes.

I tried to increase the akka.framesize configuration value but then 
realized the executePlanWithJars method in the RemoteExecutor creates a 
client with a new configuration.

Am I doing something wrong here or how can I provide my own 
configuration to the Client?

Best
Andreas Kunft

Re: Exception during job submission in RemoteEnvironment

[Till Rohrmann <tr...@apache.org>]

Hi Andreas,

for me this looks like an error. The problem is that the Client does not
respect the global configuration file when you execute your program from
Java/IDE. There is also no way to specify a configuration file for the
RemoteEnvironment, unlike for the LocalEnvironment. I think this should be
added.

Can you file an issue for that?

Cheers,
Till
​

On Thu, Jul 16, 2015 at 6:01 PM, Andreas Kunft <an...@tu-berlin.de>
wrote:

> Hello,
>
> I want to execute a Job from Java on an already running instance of Flink.
> I connect via the RemoteEnvironment. Everything goes fine, but then I get
> an exception from akka telling me the payload size is exceeded:
>
> 15-07-16 17:47:27 [ERROR] Transient association error (association remains
> live)
> akka.remote.OversizedPayloadException: Discarding oversized payload sent
> to Actor[akka.tcp://flink@127.0.0.1:6123/user/jobmanager#697056301]: max
> allowed size 10485760 bytes, actual size of encoded class
> org.apache.flink.runtime.messages.JobManagerMessages$SubmitJob was 28318303
> bytes.
>
> I tried to increase the akka.framesize configuration value but then
> realized the executePlanWithJars method in the RemoteExecutor creates a
> client with a new configuration.
>
> Am I doing something wrong here or how can I provide my own configuration
> to the Client?
>
> Best
> Andreas Kunft
>
>
>