You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by 杨光 <la...@gmail.com> on 2017/12/14 11:35:13 UTC
Flink 1.4.0 can not override JAVA_HOME for single-job deployment on YARN
Hi,
I am usring flink single-job mode on YARN. After i upgrade flink
verson from 1.3.2 to 1.4.0, the parameter
"yarn.taskmanager.env.JAVA_HOME" doesn’t work as before.
I can only found error log on yarn like this:
Exception in thread "main" java.lang.UnsupportedClassVersionError:
org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Is there something different i should know to avoid this problem ?
Thanks!
Re: Flink 1.4.0 can not override JAVA_HOME for single-job deployment
on YARN
Posted by 杨光 <la...@gmail.com>.
Hi Stephan
I tested my case again and found the parameters "-yD
containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121" and "-yD
yarn.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " can both solve
the JRE8 error . Looks like the "yarn.taskmanager.env." is already
backwards . So my early problem is caused by the low version java for
the jobmanager process only . Do you think the issue is still
necessary?
Thanks .
2017-12-15 19:32 GMT+08:00 Stephan Ewen <se...@apache.org>:
> Could you open an issue to add the old config keys as backwards supported
> "deprecated keys"? That should help making the transition smoother.
>
> On Fri, Dec 15, 2017 at 9:29 AM, Fabian Hueske <fh...@gmail.com> wrote:
>>
>> Thanks for reporting back!
>>
>> 2017-12-15 4:52 GMT+01:00 杨光 <la...@gmail.com>:
>>>
>>> Yes , i'm using Java8 , and i found the 1.4 version provided new
>>> parameters : "containerized.master.env.ENV_VAR1" and
>>> "containerized.taskmanager.env".
>>> I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
>>> " -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
>>> containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it works
>>> .
>>> Thanks a lot .
>>>
>>> 2017-12-14 20:52 GMT+08:00 Nico Kruber <ni...@data-artisans.com>:
>>> > Hi,
>>> > are you running Flink in an JRE >= 8? We dropped Java 7 support for
>>> > Flink 1.4.
>>> >
>>> >
>>> > Nico
>>> >
>>> > On 14/12/17 12:35, 杨光 wrote:
>>> >> Hi,
>>> >> I am usring flink single-job mode on YARN. After i upgrade flink
>>> >> verson from 1.3.2 to 1.4.0, the parameter
>>> >> "yarn.taskmanager.env.JAVA_HOME" doesn’t work as before.
>>> >> I can only found error log on yarn like this:
>>> >>
>>> >> Exception in thread "main" java.lang.UnsupportedClassVersionError:
>>> >> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
>>> >> major.minor version 52.0
>>> >> at java.lang.ClassLoader.defineClass1(Native Method)
>>> >> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>> >> at
>>> >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>> >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>> >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> >> at java.security.AccessController.doPrivileged(Native Method)
>>> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> >> at
>>> >> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>>> >>
>>> >> Is there something different i should know to avoid this problem ?
>>> >> Thanks!
>>> >>
>>> >
>>
>>
>
Re: Flink 1.4.0 can not override JAVA_HOME for single-job deployment
on YARN
Posted by Stephan Ewen <se...@apache.org>.
Could you open an issue to add the old config keys as backwards supported
"deprecated keys"? That should help making the transition smoother.
On Fri, Dec 15, 2017 at 9:29 AM, Fabian Hueske <fh...@gmail.com> wrote:
> Thanks for reporting back!
>
> 2017-12-15 4:52 GMT+01:00 杨光 <la...@gmail.com>:
>
>> Yes , i'm using Java8 , and i found the 1.4 version provided new
>> parameters : "containerized.master.env.ENV_VAR1" and
>> "containerized.taskmanager.env".
>> I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
>> " -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
>> containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it works
>> .
>> Thanks a lot .
>>
>> 2017-12-14 20:52 GMT+08:00 Nico Kruber <ni...@data-artisans.com>:
>> > Hi,
>> > are you running Flink in an JRE >= 8? We dropped Java 7 support for
>> > Flink 1.4.
>> >
>> >
>> > Nico
>> >
>> > On 14/12/17 12:35, 杨光 wrote:
>> >> Hi,
>> >> I am usring flink single-job mode on YARN. After i upgrade flink
>> >> verson from 1.3.2 to 1.4.0, the parameter
>> >> "yarn.taskmanager.env.JAVA_HOME" doesn’t work as before.
>> >> I can only found error log on yarn like this:
>> >>
>> >> Exception in thread "main" java.lang.UnsupportedClassVersionError:
>> >> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
>> >> major.minor version 52.0
>> >> at java.lang.ClassLoader.defineClass1(Native Method)
>> >> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> >> at java.security.SecureClassLoader.defineClass(SecureClassLoade
>> r.java:142)
>> >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> >> at java.security.AccessController.doPrivileged(Native Method)
>> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> >> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.
>> java:482)
>> >>
>> >> Is there something different i should know to avoid this problem ?
>> >> Thanks!
>> >>
>> >
>>
>
>
Re: Flink 1.4.0 can not override JAVA_HOME for single-job deployment
on YARN
Posted by Fabian Hueske <fh...@gmail.com>.
Thanks for reporting back!
2017-12-15 4:52 GMT+01:00 杨光 <la...@gmail.com>:
> Yes , i'm using Java8 , and i found the 1.4 version provided new
> parameters : "containerized.master.env.ENV_VAR1" and
> "containerized.taskmanager.env".
> I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
> " -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
> containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it works
> .
> Thanks a lot .
>
> 2017-12-14 20:52 GMT+08:00 Nico Kruber <ni...@data-artisans.com>:
> > Hi,
> > are you running Flink in an JRE >= 8? We dropped Java 7 support for
> > Flink 1.4.
> >
> >
> > Nico
> >
> > On 14/12/17 12:35, 杨光 wrote:
> >> Hi,
> >> I am usring flink single-job mode on YARN. After i upgrade flink
> >> verson from 1.3.2 to 1.4.0, the parameter
> >> "yarn.taskmanager.env.JAVA_HOME" doesn’t work as before.
> >> I can only found error log on yarn like this:
> >>
> >> Exception in thread "main" java.lang.UnsupportedClassVersionError:
> >> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
> >> major.minor version 52.0
> >> at java.lang.ClassLoader.defineClass1(Native Method)
> >> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> >> at java.security.SecureClassLoader.defineClass(
> SecureClassLoader.java:142)
> >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >> at java.security.AccessController.doPrivileged(Native Method)
> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >> at sun.launcher.LauncherHelper.checkAndLoadMain(
> LauncherHelper.java:482)
> >>
> >> Is there something different i should know to avoid this problem ?
> >> Thanks!
> >>
> >
>
Re: Flink 1.4.0 can not override JAVA_HOME for single-job deployment
on YARN
Posted by 杨光 <la...@gmail.com>.
Yes , i'm using Java8 , and i found the 1.4 version provided new
parameters : "containerized.master.env.ENV_VAR1" and
"containerized.taskmanager.env".
I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
" -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it works
.
Thanks a lot .
2017-12-14 20:52 GMT+08:00 Nico Kruber <ni...@data-artisans.com>:
> Hi,
> are you running Flink in an JRE >= 8? We dropped Java 7 support for
> Flink 1.4.
>
>
> Nico
>
> On 14/12/17 12:35, 杨光 wrote:
>> Hi,
>> I am usring flink single-job mode on YARN. After i upgrade flink
>> verson from 1.3.2 to 1.4.0, the parameter
>> "yarn.taskmanager.env.JAVA_HOME" doesn’t work as before.
>> I can only found error log on yarn like this:
>>
>> Exception in thread "main" java.lang.UnsupportedClassVersionError:
>> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
>> major.minor version 52.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>>
>> Is there something different i should know to avoid this problem ?
>> Thanks!
>>
>
Re: Flink 1.4.0 can not override JAVA_HOME for single-job deployment
on YARN
Posted by Nico Kruber <ni...@data-artisans.com>.
Hi,
are you running Flink in an JRE >= 8? We dropped Java 7 support for
Flink 1.4.
Nico
On 14/12/17 12:35, 杨光 wrote:
> Hi,
> I am usring flink single-job mode on YARN. After i upgrade flink
> verson from 1.3.2 to 1.4.0, the parameter
> "yarn.taskmanager.env.JAVA_HOME" doesn’t work as before.
> I can only found error log on yarn like this:
>
> Exception in thread "main" java.lang.UnsupportedClassVersionError:
> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
> major.minor version 52.0
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>
> Is there something different i should know to avoid this problem ?
> Thanks!
>