You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by Jin Sun <is...@gmail.com> on 2018/10/31 00:55:01 UTC

Run Flink Natively On Kubernetes

Hi all,

FLIP-6 outlines improvements in how Flink is used on top of various cluster managers. we already have implementation on Yarn and Mesos, however, no native implementation on K8s. As a follow up, FLINK-9953 <https://issues.apache.org/jira/browse/FLINK-9953> was opened by Till to track this issue, which proposal the native K8s integfation. 

Compare to the the basically flink on K8s, which provisioning a static cluster, Flink on native K8s support  dynamic allocation, resource isolation and quota management, and better user experience. 

Here is a initial design <https://docs.google.com/document/d/1Zmhui_29VASPcBOEqyMWnF3L6WEWZ4kedrCqya0WaAk/edit?usp=sharing> document contains more details of the proposal, comments are welcomed. 

Jin

Re: Demo of the K8S Flink Operator at the K8s Big Data SIG

Posted by Anand Swaminathan <as...@lyft.com.INVALID>.

Thank you all for providing us the opportunity to present our work.

Here is the presentation
<https://docs.google.com/presentation/d/10YH0vOW7juQGvXkPK-kXkNN4hj4NveI6VdpP29ezoSo/edit#slide=id.g457fba9765_0_454>
that we used.

Also, you can find more information about FlinkK8sOperator here
<https://docs.google.com/document/d/1_AITTq0fay-NwUIqNfqN0pum-erOmS_4kYZnlMxw6_o/edit#>

Please reach out us for more information. We are happy to collaborate with
you.

Thanks,
Anand

On Wed, Oct 31, 2018 at 1:54 PM Erik Erlandson <ee...@redhat.com> wrote:

> This week at the K8S Big Data SIG we had a demo of a Flink K8s operator
> from Anand Swaminathan and Ketan Umare of Lyft. The talk was recorded, and
> the link is here:
>
> https://zoom.us/recording/share/T-MWuMwfY-6S3Y4lfu9lrLYOQwJGQmxdsBlFZlNws7CwIumekTziMw?startTime=1541005448000
>
> Thanks Anand and Ketan for the great talk and demo!
> Cheers,
> Erik Erlandson
>

Re: Run Flink Natively On Kubernetes

Posted by Till Rohrmann <tr...@apache.org>.

Thanks for sharing this design document with the community. I really like
the idea of having a better K8s integration in Flink. This will allow
people to use Flink more easily in different environments.

Cheers,
Till

On Wed, Oct 31, 2018 at 1:55 AM Jin Sun <is...@gmail.com> wrote:

> Hi all,
>
> FLIP-6 outlines improvements in how Flink is used on top of various
> cluster managers. we already have implementation on Yarn and Mesos,
> however, no native implementation on K8s. As a follow up, FLINK-9953
> <https://issues.apache.org/jira/browse/FLINK-9953> was opened by Till to
> track this issue, which proposal the native K8s integfation.
>
> Compare to the the basically flink on K8s, which provisioning a static
> cluster, Flink on native K8s support  dynamic allocation, resource
> isolation and quota management, and better user experience.
>
> Here is a initial design
> <https://docs.google.com/document/d/1Zmhui_29VASPcBOEqyMWnF3L6WEWZ4kedrCqya0WaAk/edit?usp=sharing> document
> contains more details of the proposal, comments are welcomed.
>
> Jin
>