You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Bijay Pathak <bi...@cloudwick.com> on 2015/04/17 02:10:48 UTC

Base metrics for Spark Benchmarking.

Hello,

We wanted to tune the Spark running on YARN cluster.The Spark History
Server UI shows lots of parameters like:

   - GC time
   - Task Duration
   - Shuffle R/W
   - Shuffle Spill (Memory/Disk)
   - Serialization Time (Task/Result)
   - Scheduler Delay

Among the above metrics, which are the most important that should be taken
as reference for benchmarking the cluster performance?

Thanks,

Bijay