You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Paul Lam <pa...@gmail.com> on 2018/08/29 07:27:13 UTC
Custom java home on YARN
Hi,
I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.
Am I missing something? And should I create an issue requesting for that?
Best,
Paul Lam
Re: Custom java home on YARN
Posted by vino yang <ya...@gmail.com>.
Glad to hear this message!
Paul Lam <pa...@gmail.com> 于2018年8月30日周四 上午10:35写道：
> Hi vino,
>
> Yes, I’m trying to set Java 8 environment for Flink job, but YARN is
> running on Java 7 which I can’t change.
>
> I find the solution in the mail list you referred to. It turns out that I
> was using a wrong key to set Java home for the jobmanager, and the right
> key should be “containerized.master.env.JAVA_HOME”.
>
> Thank you very much!
>
> Best,
> Paul Lam
>
> 在 2018年8月30日，10:09，vino yang <ya...@gmail.com> 写道：
>
> Hi Paul,
>
> This exception means that the jdk version of the execution code is lower
> than the compiled jdk version.
> 52 means that the JDK version that compiles it is 1.8.
> There is information that Flink 1.4 has removed support for Java 7, so
> your current best choice is to use JDK 1.8.[1]
>
> [1]:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html
>
> Thanks, vino.
>
> Paul Lam <pa...@gmail.com> 于2018年8月30日周四 上午9:58写道：
>
>> Hi vino,
>>
>> Thanks for your suggestion. I believe it’s working. But I still get this
>> exception in jobmanager.err:
>>
>> > java.lang.UnsupportedClassVersionError:
>> org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported
>> major.minor version 52.0
>>
>> I tried setting Java home for the jobmanager too, but it doesn’t help.
>>
>> Do you have any idea on this? Thanks a lot!
>>
>> Best,
>> Paul Lam
>>
>>
>> 在 2018年8月29日，16:08，vino yang <ya...@gmail.com> 写道：
>>
>> Hi Paul,
>>
>> You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.
>>
>> Thanks, vino.
>>
>> Paul Lam <pa...@gmail.com> 于2018年8月29日周三 下午3:27写道：
>>
>>> Hi,
>>>
>>> I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I
>>> think it’s a quite common scenario, but it seems that currently there’s no
>>> way to pass the JAVA_HOME environment variable to YARN.
>>>
>>> Am I missing something? And should I create an issue requesting for that?
>>>
>>> Best,
>>> Paul Lam
>>
>>
>>
>
Re: Custom java home on YARN
Posted by Paul Lam <pa...@gmail.com>.
Hi tison,
This information is very helpful. Thank you!
Best,
Paul Lam
> 在 2018年8月30日，10:41，陈梓立 <wa...@gmail.com> 写道：
>
> Hi Paul,
>
> For your information, `yarn.taskmanager.env.JAVA_HOME` is deprecated, you can set env of taskmanager using the key similar to jobmanager, that is `containerized.taskmanager.env.JAVA_HOME`
> All args with prefix `containerized.taskmanager.env.` would be passed to taskmanager, similar to jobmanager.
>
> Best,
> tison.
>
>
> Paul Lam <paullin3280@gmail.com <ma...@gmail.com>> 于2018年8月30日周四 上午10:35写道：
> Hi vino,
>
> Yes, I’m trying to set Java 8 environment for Flink job, but YARN is running on Java 7 which I can’t change.
>
> I find the solution in the mail list you referred to. It turns out that I was using a wrong key to set Java home for the jobmanager, and the right key should be “containerized.master.env.JAVA_HOME”.
>
> Thank you very much!
>
> Best,
> Paul Lam
>
>> 在 2018年8月30日，10:09，vino yang <yanghua1127@gmail.com <ma...@gmail.com>> 写道：
>>
>> Hi Paul,
>>
>> This exception means that the jdk version of the execution code is lower than the compiled jdk version.
>> 52 means that the JDK version that compiles it is 1.8.
>> There is information that Flink 1.4 has removed support for Java 7, so your current best choice is to use JDK 1.8.[1]
>>
>> [1]: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html>
>>
>> Thanks, vino.
>>
>> Paul Lam <paullin3280@gmail.com <ma...@gmail.com>> 于2018年8月30日周四 上午9:58写道：
>> Hi vino,
>>
>> Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:
>>
>> > java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0
>>
>> I tried setting Java home for the jobmanager too, but it doesn’t help.
>>
>> Do you have any idea on this? Thanks a lot!
>>
>> Best,
>> Paul Lam
>>
>>
>>> 在 2018年8月29日，16:08，vino yang <yanghua1127@gmail.com <ma...@gmail.com>> 写道：
>>>
>>> Hi Paul,
>>>
>>> You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.
>>>
>>> Thanks, vino.
>>>
>>> Paul Lam <paullin3280@gmail.com <ma...@gmail.com>> 于2018年8月29日周三 下午3:27写道：
>>> Hi,
>>>
>>> I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.
>>>
>>> Am I missing something? And should I create an issue requesting for that?
>>>
>>> Best,
>>> Paul Lam
>>
>
Re: Custom java home on YARN
Posted by Paul Lam <pa...@gmail.com>.
Hi vino,
Yes, I’m trying to set Java 8 environment for Flink job, but YARN is running on Java 7 which I can’t change.
I find the solution in the mail list you referred to. It turns out that I was using a wrong key to set Java home for the jobmanager, and the right key should be “containerized.master.env.JAVA_HOME”.
Thank you very much!
Best,
Paul Lam
> 在 2018年8月30日，10:09，vino yang <ya...@gmail.com> 写道：
>
> Hi Paul,
>
> This exception means that the jdk version of the execution code is lower than the compiled jdk version.
> 52 means that the JDK version that compiles it is 1.8.
> There is information that Flink 1.4 has removed support for Java 7, so your current best choice is to use JDK 1.8.[1]
>
> [1]: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html>
>
> Thanks, vino.
>
> Paul Lam <paullin3280@gmail.com <ma...@gmail.com>> 于2018年8月30日周四 上午9:58写道：
> Hi vino,
>
> Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:
>
> > java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0
>
> I tried setting Java home for the jobmanager too, but it doesn’t help.
>
> Do you have any idea on this? Thanks a lot!
>
> Best,
> Paul Lam
>
>
>> 在 2018年8月29日，16:08，vino yang <yanghua1127@gmail.com <ma...@gmail.com>> 写道：
>>
>> Hi Paul,
>>
>> You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.
>>
>> Thanks, vino.
>>
>> Paul Lam <paullin3280@gmail.com <ma...@gmail.com>> 于2018年8月29日周三 下午3:27写道：
>> Hi,
>>
>> I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.
>>
>> Am I missing something? And should I create an issue requesting for that?
>>
>> Best,
>> Paul Lam
>
Re: Custom java home on YARN
Posted by vino yang <ya...@gmail.com>.
Hi Paul,
This exception means that the jdk version of the execution code is lower
than the compiled jdk version.
52 means that the JDK version that compiles it is 1.8.
There is information that Flink 1.4 has removed support for Java 7, so your
current best choice is to use JDK 1.8.[1]
[1]:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html
Thanks, vino.
Paul Lam <pa...@gmail.com> 于2018年8月30日周四 上午9:58写道：
> Hi vino,
>
> Thanks for your suggestion. I believe it’s working. But I still get this
> exception in jobmanager.err:
>
> > java.lang.UnsupportedClassVersionError:
> org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported
> major.minor version 52.0
>
> I tried setting Java home for the jobmanager too, but it doesn’t help.
>
> Do you have any idea on this? Thanks a lot!
>
> Best,
> Paul Lam
>
>
> 在 2018年8月29日，16:08，vino yang <ya...@gmail.com> 写道：
>
> Hi Paul,
>
> You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.
>
> Thanks, vino.
>
> Paul Lam <pa...@gmail.com> 于2018年8月29日周三 下午3:27写道：
>
>> Hi,
>>
>> I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I
>> think it’s a quite common scenario, but it seems that currently there’s no
>> way to pass the JAVA_HOME environment variable to YARN.
>>
>> Am I missing something? And should I create an issue requesting for that?
>>
>> Best,
>> Paul Lam
>
>
>
Re: Custom java home on YARN
Posted by Paul Lam <pa...@gmail.com>.
Hi vino,
Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:
> java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0
I tried setting Java home for the jobmanager too, but it doesn’t help.
Do you have any idea on this? Thanks a lot!
Best,
Paul Lam
> 在 2018年8月29日，16:08，vino yang <ya...@gmail.com> 写道：
>
> Hi Paul,
>
> You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.
>
> Thanks, vino.
>
> Paul Lam <paullin3280@gmail.com <ma...@gmail.com>> 于2018年8月29日周三 下午3:27写道：
> Hi,
>
> I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.
>
> Am I missing something? And should I create an issue requesting for that?
>
> Best,
> Paul Lam
Re: Custom java home on YARN
Posted by vino yang <ya...@gmail.com>.
Hi Paul,
You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.
Thanks, vino.
Paul Lam <pa...@gmail.com> 于2018年8月29日周三 下午3:27写道：
> Hi,
>
> I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I
> think it’s a quite common scenario, but it seems that currently there’s no
> way to pass the JAVA_HOME environment variable to YARN.
>
> Am I missing something? And should I create an issue requesting for that?
>
> Best,
> Paul Lam