You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by nilmish <ni...@gmail.com> on 2014/06/04 10:45:18 UTC

Problem understanding log message in SparkStreaming

I wanted to know the meaning of the following log message when running a
spark streaming job :

[spark-akka.actor.default-dispatcher-18] INFO 
org.apache.spark.streaming.scheduler.JobScheduler - Total delay: 5.432 s for
time 1401870454500 ms (execution: 0.593 s)

According to my understanding, total delay here means total end-to-end delay
which is here 5.432 sec. 

What is the meaning of execution : 0.593 ??

Is it the time taken for executing this particular query ?

PS : I am running a streaming job over a window of 5 mins and quering every
1.5 sec.




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Problem-understanding-log-message-in-SparkStreaming-tp6893.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.