Continue batch job with streaming job

[Sergei Poganshev <s....@slice.com>]

Is there a way to make a checkpoint/savepoint after the batch job has
finished and then run the job in a streaming mode with state that has been
initialized in batch mode?

Or more generally speaking, what are the battle-tested solutions to "job
initialization" problem, especially when there are terabytes of data that
have to be processed before the job can be turned into streaming mode? Do
any examples exist?

Re: Continue batch job with streaming job

[Andrey Zagrebin <an...@data-artisans.com>]

Hi Sergei,

Batch mode does not support state.
At the moment there is no “out of the box” way to bootstrap the state for streaming job.
You can have a look at Bravo tool [1] which is developed to address this problem.

Best,
Andrey

[1] https://github.com/king/bravo

> On 29 Oct 2018, at 22:29, Sergei Poganshev <s....@slice.com> wrote:
> 
> Is there a way to make a checkpoint/savepoint after the batch job has finished and then run the job in a streaming mode with state that has been initialized in batch mode?
> 
> Or more generally speaking, what are the battle-tested solutions to "job initialization" problem, especially when there are terabytes of data that have to be processed before the job can be turned into streaming mode? Do any examples exist?