Scheduler delay vs. Getting result time

[Hans van den Bogert <ha...@gmail.com>]

Hi, 

In the Spark UI, under “Show additional metrics”, there are two extra metrics you can show 
.1 Scheduler delay
.2 and Getting result time

When hovering “Scheduler Delay" it says (among other things):
…time to send task result from executor…

When hovering “Getting result time”:
Time that the driver spends fetching task results from workers.

What are the differences between the two?

In my case I’m benchmarking with some sleep commands and returning some big arrays, per task, to emulate execution time and network communication respectively. I can’t see any “Getting Result Time” increases, they are simple 0ms. I’m using a ‘collect’ command and can see the synthetic result arrays when I use a spark-shell.

Regards,

Hans



---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org