You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by SRK <sw...@gmail.com> on 2017/06/27 18:36:30 UTC
How do I find the time taken by each step in a stage in a Spark Job
Hi,
How do I find the time taken by each step in a stage in spark job? Also, how
do I find the bottleneck in each step and if a stage is skipped because of
the RDDs being persisted in streaming?
I am trying to identify which step in a job is taking time in my Streaming
job.
Thanks!
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-do-I-find-the-time-taken-by-each-step-in-a-stage-in-a-Spark-Job-tp28796.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: How do I find the time taken by each step in a stage in a Spark Job
Posted by swetha kasireddy <sw...@gmail.com>.
Yes, the Spark UI has some information but, it's not that helpful to find
out which particular stage is taking time.
On Wed, Jun 28, 2017 at 12:51 AM, 萝卜丝炒饭 <14...@qq.com> wrote:
> You can find the information from the spark UI
>
> ---Original---
> *From:* "SRK"<sw...@gmail.com>
> *Date:* 2017/6/28 02:36:37
> *To:* "user"<us...@spark.apache.org>;
> *Subject:* How do I find the time taken by each step in a stage in a
> Spark Job
>
> Hi,
>
> How do I find the time taken by each step in a stage in spark job? Also,
> how
> do I find the bottleneck in each step and if a stage is skipped because of
> the RDDs being persisted in streaming?
>
> I am trying to identify which step in a job is taking time in my Streaming
> job.
>
> Thanks!
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/How-do-I-find-the-time-taken-by-each-
> step-in-a-stage-in-a-Spark-Job-tp28796.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
Re: How do I find the time taken by each step in a stage in a Spark Job
Posted by 萝卜丝炒饭 <14...@qq.com>.
You can find the information from the spark UI
---Original---
From: "SRK"<sw...@gmail.com>
Date: 2017/6/28 02:36:37
To: "user"<us...@spark.apache.org>;
Subject: How do I find the time taken by each step in a stage in a Spark Job
Hi,
How do I find the time taken by each step in a stage in spark job? Also, how
do I find the bottleneck in each step and if a stage is skipped because of
the RDDs being persisted in streaming?
I am trying to identify which step in a job is taking time in my Streaming
job.
Thanks!
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-do-I-find-the-time-taken-by-each-step-in-a-stage-in-a-Spark-Job-tp28796.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org