You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Nikhil Goyal <no...@gmail.com> on 2023/05/31 22:19:06 UTC

Viewing UI for spark jobs running on K8s

Hi folks,
Is there an equivalent of the Yarn RM page for Spark on Kubernetes. We can
port-forward the UI from the driver pod for each but this process is
tedious given we have multiple jobs running. Is there a clever solution to
exposing all Driver UIs in a centralized place?

Thanks
Nikhil

Re: Viewing UI for spark jobs running on K8s

Posted by Qian Sun <qi...@gmail.com>.

Hi Nikhil

Spark operator supports ingress for exposing all UIs of running spark
applications.

reference:
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/quick-start-guide.md#driver-ui-access-and-ingress

On Thu, Jun 1, 2023 at 6:19 AM Nikhil Goyal <no...@gmail.com> wrote:

> Hi folks,
> Is there an equivalent of the Yarn RM page for Spark on Kubernetes. We can
> port-forward the UI from the driver pod for each but this process is
> tedious given we have multiple jobs running. Is there a clever solution to
> exposing all Driver UIs in a centralized place?
>
> Thanks
> Nikhil
>

-- 
Regards,
Qian Sun