You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by emceemouli <ch...@calgary.ca> on 2017/07/28 16:03:52 UTC

Re: Spark Streaming with long batch / window duration

Thanks. If i not use Window and choose to use Streaming the data on to HDFS,
could you suggest how to only store 1 week worth of data. Should i create a
cron job to delete HDFS files older than a week. PLease let me know if you
have any other suggestions



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-with-long-batch-window-duration-tp10191p29005.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.