You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Ninja Coder <ni...@gmail.com> on 2019/12/31 00:16:03 UTC
'ExecutorTaskSummary' alternative in Spark 2.3 onwards
I have a spark streaming application (currently spark 2.2) which is using
`org.apache.spark.ui.exec.ExecutorTaskSummary` to grab executor details like
duration, tasks failed, tasks completed, GC time, etc after each batch is
completed. These metrics are then loaded to prometheus and pagerduty alerts
are set on it.
We are planning to upgrade spark and looks like`ExecutorTaskSummary` is no
longer available from spark 2.3
I would like to know what are the other alternatives which I can use.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: 'ExecutorTaskSummary' alternative in Spark 2.3 onwards
Posted by Ninja Coder <ni...@gmail.com>.
Any advise/help here is much appreciated.
On Mon, Dec 30, 2019 at 4:16 PM Ninja Coder <ni...@gmail.com> wrote:
> I have a spark streaming application (currently spark 2.2) which is using
> `org.apache.spark.ui.exec.ExecutorTaskSummary` to grab executor details
> like
> duration, tasks failed, tasks completed, GC time, etc after each batch is
> completed. These metrics are then loaded to prometheus and pagerduty alerts
> are set on it.
>
> We are planning to upgrade spark and looks like`ExecutorTaskSummary` is no
> longer available from spark 2.3
>
> I would like to know what are the other alternatives which I can use.
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>