Starting beam pipeline from savepoint

[borisbolvig <bo...@boc.dk>]

Is it possible to start Beam pipelines from savepoints when running on Flink
(v.1.4)?

I am running flink in a remote environment, and executing the pipelines as
.jars specifying the flink jobmanager via cmd line arguments. This is
opposed to passing the .jar to `flink run`.

In this way the jar files are not directly available to the jobmanager, and
indeed the API /jars returns empty. 

Now to restart from a savepoint (either using the REST API or CLI) it is
expected that I pass the pipeline jar. However this is not available to the
jobmanager.

Note that automatic restarting from checkpoints and creating savepoints is
working.

Thank you for your help.





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Starting beam pipeline from savepoint

[Piotr Nowojski <pi...@data-artisans.com>]

Hi,

I am not sure, but it probably brings down to the question, whether you can restore from a save point when using RemoteEnvironment - and answer for this question is unfortunately no.

Maybe Aljoscha will know something more.

Piotrek

> On 23 May 2018, at 19:25, borisbolvig <bo...@boc.dk> wrote:
> 
> Is it possible to start Beam pipelines from savepoints when running on Flink
> (v.1.4)?
> 
> I am running flink in a remote environment, and executing the pipelines as
> .jars specifying the flink jobmanager via cmd line arguments. This is
> opposed to passing the .jar to `flink run`.
> 
> In this way the jar files are not directly available to the jobmanager, and
> indeed the API /jars returns empty. 
> 
> Now to restart from a savepoint (either using the REST API or CLI) it is
> expected that I pass the pipeline jar. However this is not available to the
> jobmanager.
> 
> Note that automatic restarting from checkpoints and creating savepoints is
> working.
> 
> Thank you for your help.
> 
> 
> 
> 
> 
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/