Re: Uploaded jar disappears when web monitor restarts

[Till Rohrmann <tr...@apache.org>]

Hi Emanuele,

you're right, the upload directory for the jars is deleted whenever the
WebRuntimeMonitor is stopped and a new directory is created when the
WebRuntimeMonitor is started.

 It would be doable to introduce a configuration parameter for the uploaded
jar directory so that you don't have to upload the jars every time again.
But then the user would have to responsible for cleaning it up. If you
want, then you can open a JIRA issue for this feature.

I would recommend using the CLI for starting Flink jobs. The reason is that
the WebRuntimeMonitor does not support jobs which contain eager execution
trigger calls such as collect, count and print, for example.

Cheers,
Till

On Wed, Jun 8, 2016 at 5:06 AM, Emanuele Cesena <em...@shopkick.com>
wrote:

> Hi,
>
> When the web monitor restarts the uploaded jars disappear — in fact, every
> time it restarts the upload directory is different.
>
> This seems intentional:
>
> https://github.com/apache/flink/blob/master/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java#L162
>
> Could anyone confirm?
>
> Wouldn’t it be useful to have a config param to be able to set it
> permanently (and thus avoiding jars to be deleted)? Or, what is the
> intended way to “add jars to my cluster”?
>
> Thank you,
> E.
>
>

Re: Uploaded jar disappears when web monitor restarts

[Emanuele Cesena <em...@shopkick.com>]

Thanks for the clarifications (and for the note on eager exec, I’ll read more about it).

I’ll open a JIRA ticket then.

Best,


> On Jun 8, 2016, at 1:57 AM, Till Rohrmann <tr...@apache.org> wrote:
> 
> Hi Emanuele,
> 
> you're right, the upload directory for the jars is deleted whenever the WebRuntimeMonitor is stopped and a new directory is created when the WebRuntimeMonitor is started.
> 
>  It would be doable to introduce a configuration parameter for the uploaded jar directory so that you don't have to upload the jars every time again. But then the user would have to responsible for cleaning it up. If you want, then you can open a JIRA issue for this feature.
> 
> I would recommend using the CLI for starting Flink jobs. The reason is that the WebRuntimeMonitor does not support jobs which contain eager execution trigger calls such as collect, count and print, for example.
> 
> Cheers,
> Till
> 
> On Wed, Jun 8, 2016 at 5:06 AM, Emanuele Cesena <em...@shopkick.com> wrote:
> Hi,
> 
> When the web monitor restarts the uploaded jars disappear — in fact, every time it restarts the upload directory is different.
> 
> This seems intentional:
> https://github.com/apache/flink/blob/master/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java#L162
> 
> Could anyone confirm?
> 
> Wouldn’t it be useful to have a config param to be able to set it permanently (and thus avoiding jars to be deleted)? Or, what is the intended way to “add jars to my cluster”?
> 
> Thank you,
> E.
> 
>