You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Josh J <jo...@gmail.com> on 2015/01/31 04:59:14 UTC

measuring time taken in map, reduceByKey, filter, flatMap

Hi,

I have a stream pipeline which invokes map, reduceByKey, filter, and
flatMap. How can I measure the time taken in each stage?

Thanks,
Josh

Re: measuring time taken in map, reduceByKey, filter, flatMap

Posted by Akhil Das <ak...@sigmoidanalytics.com>.

I believe From the webui (running on port 8080) you will get these
measurements.

Thanks
Best Regards

On Sat, Jan 31, 2015 at 9:29 AM, Josh J <jo...@gmail.com> wrote:

> Hi,
>
> I have a stream pipeline which invokes map, reduceByKey, filter, and
> flatMap. How can I measure the time taken in each stage?
>
> Thanks,
> Josh
>