You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Noah (Jira)" <ji...@apache.org> on 2020/06/18 09:44:00 UTC

[jira] [Created] (FLINK-18362) Shipping jdk by shipping archive

Noah created FLINK-18362:
----------------------------

             Summary: Shipping jdk by shipping archive
                 Key: FLINK-18362
                 URL: https://issues.apache.org/jira/browse/FLINK-18362
             Project: Flink
          Issue Type: Wish
    Affects Versions: 1.10.1
            Reporter: Noah


Hello,
Our team are running flink cluster on YARN, and it works so well 👍

h4. Functional requirements
Is there any option to ship archive to YARN applications?

h4. Backgrounds
Recently, one of job has been shut down with jdk8 version related issues.
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6675699

It's easy problem if we could set latest jdk on `containerized.taskmanager.env.JAVA_HOME`.
However, cluster administrator said it's difficult to install the latest jdk on all cluster machines.
 
So, we planned to run a job on latest jdk that is shipped via shared resources.
There's an option `yarn.ship-directories` but it's quite slow because jdk has large number of files.

If Flink supports to ship archive such as `yarn.ship-archive`, we can ship jdk archive to remote machines and use shipped jdk location as JAVA_HOME (using `yarn.container-start-command-template` )




--
This message was sent by Atlassian Jira
(v8.3.4#803005)