You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Collin Shi <sh...@aliyun.com> on 2016/01/19 04:18:35 UTC

Spark Streaming - Latest batch-time can't keep up with current time

Hi all, 

After having submit the job, the latest batch-time is almost as same as current time at first. Let's say, if current time is '12:00:00', then the latest batch-time would be '11:59:59'. But as time goes, the difference is getting greater and greater. For instance , current time is '13:00:00' but the latest batch-time is '12:40:00'. 

The total-delay is sometimes greater than batch-interval, but it won't have influence on batch-time right ?

How could I fix this problem?

I'am using spark 1.4.0 and kafka. 



Thanks,
Collin