Measure Serialization / De-serialization Time

[Jack Kolokasis <ko...@ics.forth.gr>]

Hello all,

     I am running a simple Word Count application using storage level 
MEMORY_ONLY in one case and OFF_HEAP on the other. I see that the 
execution time while I ran my application off-heap is higher than 
on-heap. So, I am looking where this time goes.

One thought at first is that this time goes on serialization and 
deserialization time of the objects.

     - I want to measure that, is there any way ?

     - Also do you have any opinion where this time goes ?

My experimental setup is : 1 driver and 1 executor (on the same 
machine). Executor has 1GB memory and 1 core. The offHeap size is 1GB.

Thanks for your reply,
--Iacovos


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org