You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by yam <yo...@playtech.com> on 2015/12/07 17:23:46 UTC

How to change StreamingContext batch duration after loading from checkpoint

Is there a way to change the streaming context batch interval after reloading
from checkpoint?

I would like to be able to change the batch interval after restarting the
application without loosing the checkpoint of course.

Thanks!




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-change-StreamingContext-batch-duration-after-loading-from-checkpoint-tp25624.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Re: How to change StreamingContext batch duration after loading from checkpoint

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Taking the values from a configuration file rather hard-coding in the code
might help, haven't tried it though.

Thanks
Best Regards

On Mon, Dec 7, 2015 at 9:53 PM, yam <yo...@playtech.com> wrote:

> Is there a way to change the streaming context batch interval after
> reloading
> from checkpoint?
>
> I would like to be able to change the batch interval after restarting the
> application without loosing the checkpoint of course.
>
> Thanks!
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-change-StreamingContext-batch-duration-after-loading-from-checkpoint-tp25624.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: How to change StreamingContext batch duration after loading from checkpoint

Posted by manasdebashiskar <po...@gmail.com>.
Not sure what is your requirement there, but if you have a 2 second streaming
batch , you can create a 4 second stream out of it but the other way is not
possible.
Basically you can create one stream out of another stream.

..Manas



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-change-StreamingContext-batch-duration-after-loading-from-checkpoint-tp25624p25680.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org