You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yang Wang (Jira)" <ji...@apache.org> on 2020/01/15 06:23:00 UTC
[jira] [Updated] (FLINK-10934) Support per-job mode in kubernetes
[ https://issues.apache.org/jira/browse/FLINK-10934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yang Wang updated FLINK-10934:
------------------------------
Description:
The Kubernetes do not have a default distributed storage and provide public api to ship files like Yarn local resource. So we could not ship the user jars and files on the client side to jobmanager and taskmanager. Also it is not a common way on Kubernetes. Instead, users usually build their jars and files into the docker image. So when the jobmanager and taskmanager are launched, the users jars already existed.
Even if some users do not want to build the jars into the image, they could use the initContainer to download the jars from the storage(http/s3/etc.).
All in all, the Kubernetes per-job cluster will only support cluster deploy-mode.
This ticket depends on FLIP-85(Support cluster deployment). Please reference the documentation [here|https://docs.google.com/document/d/1aAwVjdZByA-0CHbgv16Me-vjaaDMCfhX7TzVVTuifYM/edit?usp=sharing] for more information.
was:* Implement KubernetesJobClusterEntrypoint
> Support per-job mode in kubernetes
> ----------------------------------
>
> Key: FLINK-10934
> URL: https://issues.apache.org/jira/browse/FLINK-10934
> Project: Flink
> Issue Type: Sub-task
> Components: Deployment / Kubernetes
> Reporter: JIN SUN
> Assignee: Yang Wang
> Priority: Major
>
> The Kubernetes do not have a default distributed storage and provide public api to ship files like Yarn local resource. So we could not ship the user jars and files on the client side to jobmanager and taskmanager. Also it is not a common way on Kubernetes. Instead, users usually build their jars and files into the docker image. So when the jobmanager and taskmanager are launched, the users jars already existed.
> Even if some users do not want to build the jars into the image, they could use the initContainer to download the jars from the storage(http/s3/etc.).
> All in all, the Kubernetes per-job cluster will only support cluster deploy-mode.
>
> This ticket depends on FLIP-85(Support cluster deployment). Please reference the documentation [here|https://docs.google.com/document/d/1aAwVjdZByA-0CHbgv16Me-vjaaDMCfhX7TzVVTuifYM/edit?usp=sharing] for more information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)