You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by yam <yo...@playtech.com> on 2015/12/07 17:23:46 UTC
How to change StreamingContext batch duration after loading from
checkpoint
Is there a way to change the streaming context batch interval after reloading
from checkpoint?
I would like to be able to change the batch interval after restarting the
application without loosing the checkpoint of course.
Thanks!
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-change-StreamingContext-batch-duration-after-loading-from-checkpoint-tp25624.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: How to change StreamingContext batch duration after loading from checkpoint
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Taking the values from a configuration file rather hard-coding in the code
might help, haven't tried it though.
Thanks
Best Regards
On Mon, Dec 7, 2015 at 9:53 PM, yam <yo...@playtech.com> wrote:
> Is there a way to change the streaming context batch interval after
> reloading
> from checkpoint?
>
> I would like to be able to change the batch interval after restarting the
> application without loosing the checkpoint of course.
>
> Thanks!
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-change-StreamingContext-batch-duration-after-loading-from-checkpoint-tp25624.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: How to change StreamingContext batch duration after loading
from checkpoint
Posted by manasdebashiskar <po...@gmail.com>.
Not sure what is your requirement there, but if you have a 2 second streaming
batch , you can create a 4 second stream out of it but the other way is not
possible.
Basically you can create one stream out of another stream.
..Manas
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-change-StreamingContext-batch-duration-after-loading-from-checkpoint-tp25624p25680.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org