You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Elias Levy <fe...@gmail.com> on 2017/09/19 00:44:17 UTC
Noisy org.apache.flink.configuration.GlobalConfiguration
Is there a particular reason that GlobalConfiguration is so noisy?
The task manager log is full of "Loading configuration property" messages
from GlobalConfiguration each time there is a checkpoint. Why isn't the
configuration read once?
Re: Noisy org.apache.flink.configuration.GlobalConfiguration
Posted by Ufuk Celebi <uc...@apache.org>.
PS: To answer the question. No, I think there is no reason for this
and it shouldn't happen. :-(
On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <fe...@gmail.com> wrote:
> Is there a particular reason that GlobalConfiguration is so noisy?
>
> The task manager log is full of "Loading configuration property" messages
> from GlobalConfiguration each time there is a checkpoint. Why isn't the
> configuration read once?
Re: Noisy org.apache.flink.configuration.GlobalConfiguration
Posted by Ufuk Celebi <uc...@apache.org>.
I saw this too recently when using HadoopFileSystem for checkpoints
(HDFS or S3). I thought I had opened an issue for this, but I didn't.
Here it is: https://issues.apache.org/jira/browse/FLINK-7643
On Tue, Sep 19, 2017 at 1:28 PM, Till Rohrmann <tr...@apache.org> wrote:
> Hi Elias,
>
> which version of Flink and which state backend are you running? I tried to
> reproduce it and wasn't successful so far.
>
> We recently changed a bit how we load the GlobalConfiguration in combination
> with dynamic properties [1]. Maybe this has affected what you've reported as
> well.
>
> [1] https://issues.apache.org/jira/browse/FLINK-7269
>
> Cheers,
> Till
>
> On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <fe...@gmail.com>
> wrote:
>>
>> Is there a particular reason that GlobalConfiguration is so noisy?
>>
>> The task manager log is full of "Loading configuration property" messages
>> from GlobalConfiguration each time there is a checkpoint. Why isn't the
>> configuration read once?
>
>
Re: Noisy org.apache.flink.configuration.GlobalConfiguration
Posted by Stephan Ewen <se...@apache.org>.
Hi!
The problem should be fixed in the latest master.
https://github.com/apache/flink/commit/536675b03a5050fda9c3e1fd403818cb50dcc6ff
Release should come soon...
Best,
Stephan
On Wed, Oct 4, 2017 at 12:50 PM, Stephan Ewen <se...@apache.org> wrote:
> Will be opening a PR for file system configuring which should fix that.
>
> On Tue, Sep 19, 2017 at 5:28 PM, Elias Levy <fe...@gmail.com>
> wrote:
>
>> Till,
>>
>> Using 1.3.2 and like Ufuk mentioned, using S3 for checkpointing.
>>
>> On Tue, Sep 19, 2017 at 4:28 AM, Till Rohrmann <tr...@apache.org>
>> wrote:
>>
>>> Hi Elias,
>>>
>>> which version of Flink and which state backend are you running? I tried
>>> to reproduce it and wasn't successful so far.
>>>
>>> We recently changed a bit how we load the GlobalConfiguration in
>>> combination with dynamic properties [1]. Maybe this has affected what
>>> you've reported as well.
>>>
>>> [1] https://issues.apache.org/jira/browse/FLINK-7269
>>>
>>> Cheers,
>>> Till
>>>
>>> On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <fearsome.lucidity@gmail.com
>>> > wrote:
>>>
>>>> Is there a particular reason that GlobalConfiguration is so noisy?
>>>>
>>>> The task manager log is full of "Loading configuration property"
>>>> messages from GlobalConfiguration each time there is a checkpoint. Why
>>>> isn't the configuration read once?
>>>>
>>>
>>>
>>
>
Re: Noisy org.apache.flink.configuration.GlobalConfiguration
Posted by Stephan Ewen <se...@apache.org>.
Will be opening a PR for file system configuring which should fix that.
On Tue, Sep 19, 2017 at 5:28 PM, Elias Levy <fe...@gmail.com>
wrote:
> Till,
>
> Using 1.3.2 and like Ufuk mentioned, using S3 for checkpointing.
>
> On Tue, Sep 19, 2017 at 4:28 AM, Till Rohrmann <tr...@apache.org>
> wrote:
>
>> Hi Elias,
>>
>> which version of Flink and which state backend are you running? I tried
>> to reproduce it and wasn't successful so far.
>>
>> We recently changed a bit how we load the GlobalConfiguration in
>> combination with dynamic properties [1]. Maybe this has affected what
>> you've reported as well.
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-7269
>>
>> Cheers,
>> Till
>>
>> On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <fe...@gmail.com>
>> wrote:
>>
>>> Is there a particular reason that GlobalConfiguration is so noisy?
>>>
>>> The task manager log is full of "Loading configuration property"
>>> messages from GlobalConfiguration each time there is a checkpoint. Why
>>> isn't the configuration read once?
>>>
>>
>>
>
Re: Noisy org.apache.flink.configuration.GlobalConfiguration
Posted by Elias Levy <fe...@gmail.com>.
Till,
Using 1.3.2 and like Ufuk mentioned, using S3 for checkpointing.
On Tue, Sep 19, 2017 at 4:28 AM, Till Rohrmann <tr...@apache.org> wrote:
> Hi Elias,
>
> which version of Flink and which state backend are you running? I tried to
> reproduce it and wasn't successful so far.
>
> We recently changed a bit how we load the GlobalConfiguration in
> combination with dynamic properties [1]. Maybe this has affected what
> you've reported as well.
>
> [1] https://issues.apache.org/jira/browse/FLINK-7269
>
> Cheers,
> Till
>
> On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <fe...@gmail.com>
> wrote:
>
>> Is there a particular reason that GlobalConfiguration is so noisy?
>>
>> The task manager log is full of "Loading configuration property" messages
>> from GlobalConfiguration each time there is a checkpoint. Why isn't the
>> configuration read once?
>>
>
>
Re: Noisy org.apache.flink.configuration.GlobalConfiguration
Posted by Till Rohrmann <tr...@apache.org>.
Hi Elias,
which version of Flink and which state backend are you running? I tried to
reproduce it and wasn't successful so far.
We recently changed a bit how we load the GlobalConfiguration in
combination with dynamic properties [1]. Maybe this has affected what
you've reported as well.
[1] https://issues.apache.org/jira/browse/FLINK-7269
Cheers,
Till
On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <fe...@gmail.com>
wrote:
> Is there a particular reason that GlobalConfiguration is so noisy?
>
> The task manager log is full of "Loading configuration property" messages
> from GlobalConfiguration each time there is a checkpoint. Why isn't the
> configuration read once?
>