You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Matt Cheah <mc...@palantir.com> on 2016/11/30 06:22:04 UTC

Proposal for SPARK-18278

Hi everyone,

Kubernetes is a technology that is a key player in the cluster computing world. Currently, running Spark applications on Kubernetes requires deploying a standalone Spark cluster on the Kubernetes cluster, and then running the jobs against the standalone Spark cluster. However, there would be many benefits to running Spark on Kubernetes natively, and so SPARK-18278<https://issues.apache.org/jira/browse/SPARK-18278> has been filed to track discussion around supporting Kubernetes as a cluster manager, in addition to the existing Mesos, YARN, and standalone cluster managers.

A first draft of a proposal outlining a potential long-term plan around this feature has been attached to the JIRA ticket. Any feedback and discussion would be greatly appreciated.

Thanks,

-Matt Cheah