You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Gwenhael Pasquiers <gw...@ericsson.com> on 2015/11/16 17:30:33 UTC

MaxPermSize on yarn

Hi,

We're having some OOM permgen exceptions when running on yarn.

We're not yet sure if it is either a consequence or a cause of our crashes, but we've been trying to increase that value... And we did not find how to do it.

I've seen that the yarn-daemon.sh sets a 256m value.
It looks to me that it's also possible to customize the YarnClient JVM args, but it will only be for the client, not for the TaskManagers.

Do you know of a way to do it ?

B.R.

Gwenhaël PASQUIERS

RE: MaxPermSize on yarn

Posted by Gwenhael Pasquiers <gw...@ericsson.com>.
The option was accepted using the yaml file and it looks likes it solved our issue.

Thanks again.

From: Robert Metzger [mailto:rmetzger@apache.org]
Sent: mardi 17 novembre 2015 12:04
To: user@flink.apache.org
Subject: Re: MaxPermSize on yarn

You can also put the configuration option into the flink-conf.yaml file.

On Tue, Nov 17, 2015 at 12:01 PM, Gwenhael Pasquiers <gw...@ericsson.com>> wrote:
We tried to add : -yD env.java.opts="-XX:MaxPermSize=256m"

But we’ve got to investigate since we have to following error :

Improperly specified VM option 'MaxPermSize'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Either is the option is badly supported, or it is incorrectly passed to the VM (maybe the “-XX:” causes some issues in the scripts).

We’ll investigate.

From: Robert Metzger [mailto:rmetzger@apache.org<ma...@apache.org>]
Sent: lundi 16 novembre 2015 19:18
To: user@flink.apache.org<ma...@flink.apache.org>
Subject: Re: MaxPermSize on yarn

Hi Gwen,

there is a configuration value called "env.java.opts", that allows you to pass custom JVM args to JM and TM JVMs.

I hope that helps.



On Mon, Nov 16, 2015 at 5:30 PM, Gwenhael Pasquiers <gw...@ericsson.com>> wrote:
Hi,

We’re having some OOM permgen exceptions when running on yarn.

We’re not yet sure if it is either a consequence or a cause of our crashes, but we’ve been trying to increase that value… And we did not find how to do it.

I’ve seen that the yarn-daemon.sh sets a 256m value.
It looks to me that it’s also possible to customize the YarnClient JVM args, but it will only be for the client, not for the TaskManagers.

Do you know of a way to do it ?

B.R.

Gwenhaël PASQUIERS



Re: MaxPermSize on yarn

Posted by Robert Metzger <rm...@apache.org>.
You can also put the configuration option into the flink-conf.yaml file.

On Tue, Nov 17, 2015 at 12:01 PM, Gwenhael Pasquiers <
gwenhael.pasquiers@ericsson.com> wrote:

> We tried to add : -yD env.java.opts="-XX:MaxPermSize=256m"
>
>
>
> But we’ve got to investigate since we have to following error :
>
>
>
> Improperly specified VM option 'MaxPermSize'
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
>
>
>
> Either is the option is badly supported, or it is incorrectly passed to
> the VM (maybe the “-XX:” causes some issues in the scripts).
>
>
>
> We’ll investigate.
>
>
>
> *From:* Robert Metzger [mailto:rmetzger@apache.org]
> *Sent:* lundi 16 novembre 2015 19:18
> *To:* user@flink.apache.org
> *Subject:* Re: MaxPermSize on yarn
>
>
>
> Hi Gwen,
>
>
>
> there is a configuration value called "env.java.opts", that allows you to
> pass custom JVM args to JM and TM JVMs.
>
>
>
> I hope that helps.
>
>
>
>
>
>
>
> On Mon, Nov 16, 2015 at 5:30 PM, Gwenhael Pasquiers <
> gwenhael.pasquiers@ericsson.com> wrote:
>
> Hi,
>
>
>
> We’re having some OOM permgen exceptions when running on yarn.
>
>
>
> We’re not yet sure if it is either a consequence or a cause of our
> crashes, but we’ve been trying to increase that value… And we did not find
> how to do it.
>
>
>
> I’ve seen that the yarn-daemon.sh sets a 256m value.
>
> It looks to me that it’s also possible to customize the YarnClient JVM
> args, but it will only be for the client, not for the TaskManagers.
>
>
>
> Do you know of a way to do it ?
>
>
>
> B.R.
>
>
>
> Gwenhaël PASQUIERS
>
>
>

RE: MaxPermSize on yarn

Posted by Gwenhael Pasquiers <gw...@ericsson.com>.
We tried to add : -yD env.java.opts="-XX:MaxPermSize=256m"

But we’ve got to investigate since we have to following error :

Improperly specified VM option 'MaxPermSize'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Either is the option is badly supported, or it is incorrectly passed to the VM (maybe the “-XX:” causes some issues in the scripts).

We’ll investigate.

From: Robert Metzger [mailto:rmetzger@apache.org]
Sent: lundi 16 novembre 2015 19:18
To: user@flink.apache.org
Subject: Re: MaxPermSize on yarn

Hi Gwen,

there is a configuration value called "env.java.opts", that allows you to pass custom JVM args to JM and TM JVMs.

I hope that helps.



On Mon, Nov 16, 2015 at 5:30 PM, Gwenhael Pasquiers <gw...@ericsson.com>> wrote:
Hi,

We’re having some OOM permgen exceptions when running on yarn.

We’re not yet sure if it is either a consequence or a cause of our crashes, but we’ve been trying to increase that value… And we did not find how to do it.

I’ve seen that the yarn-daemon.sh sets a 256m value.
It looks to me that it’s also possible to customize the YarnClient JVM args, but it will only be for the client, not for the TaskManagers.

Do you know of a way to do it ?

B.R.

Gwenhaël PASQUIERS


Re: MaxPermSize on yarn

Posted by Robert Metzger <rm...@apache.org>.
Hi Gwen,

there is a configuration value called "env.java.opts", that allows you to
pass custom JVM args to JM and TM JVMs.

I hope that helps.



On Mon, Nov 16, 2015 at 5:30 PM, Gwenhael Pasquiers <
gwenhael.pasquiers@ericsson.com> wrote:

> Hi,
>
>
>
> We’re having some OOM permgen exceptions when running on yarn.
>
>
>
> We’re not yet sure if it is either a consequence or a cause of our
> crashes, but we’ve been trying to increase that value… And we did not find
> how to do it.
>
>
>
> I’ve seen that the yarn-daemon.sh sets a 256m value.
>
> It looks to me that it’s also possible to customize the YarnClient JVM
> args, but it will only be for the client, not for the TaskManagers.
>
>
>
> Do you know of a way to do it ?
>
>
>
> B.R.
>
>
>
> Gwenhaël PASQUIERS
>