You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Andrey Gourine <an...@taboola.com.INVALID> on 2023/04/30 15:38:04 UTC

Any experience with K8s Remote Shuffling Service at scale?

Hi All, I am looking for people that have experience running external
shuffling service at scale with Spark 3 and K8s

I have already tried internal shuffling service (available from spark 3)
and trying to work with Uniffle
<https://github.com/apache/incubator-uniffle> (Incubating)
Any other options?
Thank you