You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Jason Jun <ja...@gmail.com> on 2022/04/11 04:42:42 UTC

Is spark fair scheduler is for kubernete?

the official doc, https://spark.apache.org/docs/latest/job-scheduling.html,
didn't mention  that its working for kubernete cluster?

Can anyone quickly answer this?

TIA.
Jason

Re: Is spark fair scheduler is for kubernete?

Posted by Martin Grigorov <mg...@apache.org>.

Hi,

On Mon, Apr 11, 2022 at 7:43 AM Jason Jun <ja...@gmail.com> wrote:

> the official doc, https://spark.apache.org/docs/latest/job-scheduling.html,
> didn't mention  that its working for kubernete cluster?
>

You could use Volcano scheduler for more advanced setups on Kubernetes.
Here is an article explaining how to make use of the fresh integration
between Spark and Volcano in 3.3 (not yet released!) -
https://martin-grigorov.medium.com/native-integration-between-apache-spark-and-volcano-kubernetes-scheduler-488f54dbbab3

Regards,
Martin

>
> Can anyone quickly answer this?
>
> TIA.
> Jason
>