how can I make the sliding window in Spark Streaming driven by data timestamp instead of absolute time

[Aries Kong <ar...@gmail.com>]

Hi again,


It seems that the sliding window in Spark Streaming is driven by
absolute time instead of the data timestamp, can anybody

kindly explains why?

How can I make the sliding window in Spark Streaming driven by data
timestamp instead of absolute time?


Thanks!


Aries Kong

Re: how can I make the sliding window in Spark Streaming driven by data timestamp instead of absolute time

[st553 <st...@gmail.com>]

I believe I have a similar question to this. I would like to process an
offline event stream for testing/debugging. The stream is stored in a CSV
file where each row in the file has a timestamp. I would like to feed this
file into Spark Streaming and have the concept of time be driven by the
timestamp column. Has anyone does this before? I haven't seen anything in
the docs. Would like to know if this is possible in Spark Streaming. Thanks!



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/how-can-I-make-the-sliding-window-in-Spark-Streaming-driven-by-data-timestamp-instead-of-absolute-tie-tp1755p16704.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org