Regarding Global Configuration in Flink

[Janardhan Reddy <ja...@olacabs.com>]

Hi,

Is global configuration same for all jobs in a Flink cluster.

Is it a good idea to write a custom source which polls some external source
every x minutes and updates the global config. Will the config change be
propagated across all jobs?

What happens when the size of global config grows too big?  Where is the
global config stored

Re: Regarding Global Configuration in Flink

[Maximilian Michels <mx...@apache.org>]

Hi!

Are you referring to the GlobalConfiguration class? That used to be a
singleton class in Flink version < 1.2.x which would load the
configuration only once per VM, if it found a config file. It allowed
operations that could change that config after it had been loaded. It
has since then been refactored to simply load the local configuration
if requested and not store it or manipulate it in any way.

Regarding your question: No, config changes won't be propagated to
jobs. The resulting Configuration object resides on the JVM heap.
Thus, the same memory limits apply as to any regular Java object.

Cheers,
Max

On Thu, Aug 25, 2016 at 6:33 AM, Janardhan Reddy
<ja...@olacabs.com> wrote:
> Hi,
>
> Is global configuration same for all jobs in a Flink cluster.
>
> Is it a good idea to write a custom source which polls some external source
> every x minutes and updates the global config. Will the config change be
> propagated across all jobs?
>
> What happens when the size of global config grows too big?  Where is the
> global config stored