You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Till Rohrmann (JIRA)" <ji...@apache.org> on 2018/06/19 07:51:00 UTC

[jira] [Updated] (FLINK-9612) Add option for minimal artifacts being pulled in Mesos

     [ https://issues.apache.org/jira/browse/FLINK-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Till Rohrmann updated FLINK-9612:
---------------------------------
    Summary: Add option for minimal artifacts being pulled in Mesos  (was: Add option for minimal artifacts being pulled in meseos)

> Add option for minimal artifacts being pulled in Mesos
> ------------------------------------------------------
>
>                 Key: FLINK-9612
>                 URL: https://issues.apache.org/jira/browse/FLINK-9612
>             Project: Flink
>          Issue Type: Improvement
>          Components: Configuration, Docker, Mesos
>            Reporter: Addison Higham
>            Priority: Major
>
> NOTE: this assumes mesos, but this improvement could also be useful for future container deployments.
> Currently, in mesos, the FlinkDistributionOverlay  copies the entire `conf`, `bin`, and `lib` folders from the running JobManager/ResourceManager. When using docker with a pre-installed flink distribution, this is relatively inefficient as it pulls jars that are already baked into the container image.
> A new option that disables pulling most (if not all?) of the FlinkDistributionOverlay could allow for much faster and more scalable provisions of TaskManagers. As it currently stands, trying to run a few hundred TaskManagers is likely to result in poor performance in pulling all the artifacts from the MesosArtifactServer



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)