You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Plamen Paskov <pl...@next-stream.com> on 2017/12/19 14:45:51 UTC
state.checkpoints.dir not configured
Hi,
I'm trying to enable externalized checkpoints like this:
env.enableCheckpointing(1000);
CheckpointConfig checkpointConfig = env.getCheckpointConfig();
checkpointConfig.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
checkpointConfig.setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
env.setStateBackend(new FsStateBackend("file:///tmp/flink-checkpoints-data/",true));
in flink-conf.yaml i set:
state.checkpoints.dir: file:///tmp/flink-checkpoints-meta/
but when i run the application i get this error: java.lang.IllegalStateException: CheckpointConfig says to persist periodic checkpoints, but no checkpoint directory has been configured. You can configure configure one via key 'state.checkpoints.dir'.
Any suggestions?
Thanks
Re: state.checkpoints.dir not configured
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
I think what might have happened is that you had an earlier JobManager still running that had the old configuration loaded. Then you tried starting a new JobManager. Could that be the case?
Best,
Aljoscha
> On 21. Dec 2017, at 16:34, Plamen Paskov <pl...@next-stream.com> wrote:
>
> I'm sorry but i already cleaned up the logs. If i encounter the same error again i will let you know
>
>
> On 21.12.2017 17:12, Ufuk Celebi wrote:
>> Could you please share the complete logs of the initial failure? What
>> you describe in your second email should not happen ;-) If the
>> JobManager cannot bind to the port it should simply die and not
>> complain about checkpoint configuration.
>>
>> – Ufuk
>>
>> On Thu, Dec 21, 2017 at 1:21 PM, Plamen Paskov
>> <pl...@next-stream.com> wrote:
>>> I inspected the log as you suggest and found that 6123 port was used by
>>> another process. I free the port and restarted the job manager. Now
>>> everything looks fine. The error message is little misleading as the real
>>> cause is that 6123 is already bind but it says that state.checkpoints.dir is
>>> not set.
>>>
>>> Thanks
>>>
>>>
>>>
>>> On 19.12.2017 17:55, Ufuk Celebi wrote:
>>>> When the JobManager/TaskManager are starting up they log what config
>>>> they are loading. Look for lines like
>>>>
>>>> "Loading configuration property: {}, {}"
>>>>
>>>> Do you find the required configuration as part of these messages?
>>>>
>>>> – Ufuk
>>>>
>>>>
>>>> On Tue, Dec 19, 2017 at 3:45 PM, Plamen Paskov
>>>> <pl...@next-stream.com> wrote:
>>>>> Hi,
>>>>> I'm trying to enable externalized checkpoints like this:
>>>>>
>>>>> env.enableCheckpointing(1000);
>>>>> CheckpointConfig checkpointConfig = env.getCheckpointConfig();
>>>>>
>>>>> checkpointConfig.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
>>>>> checkpointConfig.setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
>>>>> env.setStateBackend(new
>>>>> FsStateBackend("file:///tmp/flink-checkpoints-data/", true));
>>>>>
>>>>> in flink-conf.yaml i set:
>>>>> state.checkpoints.dir: file:///tmp/flink-checkpoints-meta/
>>>>>
>>>>> but when i run the application i get this error:
>>>>> java.lang.IllegalStateException: CheckpointConfig says to persist
>>>>> periodic
>>>>> checkpoints, but no checkpoint directory has been configured. You can
>>>>> configure configure one via key 'state.checkpoints.dir'.
>>>>>
>>>>> Any suggestions?
>>>>>
>>>>> Thanks
>>>
>
Re: state.checkpoints.dir not configured
Posted by Plamen Paskov <pl...@next-stream.com>.
I'm sorry but i already cleaned up the logs. If i encounter the same
error again i will let you know
On 21.12.2017 17:12, Ufuk Celebi wrote:
> Could you please share the complete logs of the initial failure? What
> you describe in your second email should not happen ;-) If the
> JobManager cannot bind to the port it should simply die and not
> complain about checkpoint configuration.
>
> – Ufuk
>
> On Thu, Dec 21, 2017 at 1:21 PM, Plamen Paskov
> <pl...@next-stream.com> wrote:
>> I inspected the log as you suggest and found that 6123 port was used by
>> another process. I free the port and restarted the job manager. Now
>> everything looks fine. The error message is little misleading as the real
>> cause is that 6123 is already bind but it says that state.checkpoints.dir is
>> not set.
>>
>> Thanks
>>
>>
>>
>> On 19.12.2017 17:55, Ufuk Celebi wrote:
>>> When the JobManager/TaskManager are starting up they log what config
>>> they are loading. Look for lines like
>>>
>>> "Loading configuration property: {}, {}"
>>>
>>> Do you find the required configuration as part of these messages?
>>>
>>> – Ufuk
>>>
>>>
>>> On Tue, Dec 19, 2017 at 3:45 PM, Plamen Paskov
>>> <pl...@next-stream.com> wrote:
>>>> Hi,
>>>> I'm trying to enable externalized checkpoints like this:
>>>>
>>>> env.enableCheckpointing(1000);
>>>> CheckpointConfig checkpointConfig = env.getCheckpointConfig();
>>>>
>>>> checkpointConfig.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
>>>> checkpointConfig.setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
>>>> env.setStateBackend(new
>>>> FsStateBackend("file:///tmp/flink-checkpoints-data/", true));
>>>>
>>>> in flink-conf.yaml i set:
>>>> state.checkpoints.dir: file:///tmp/flink-checkpoints-meta/
>>>>
>>>> but when i run the application i get this error:
>>>> java.lang.IllegalStateException: CheckpointConfig says to persist
>>>> periodic
>>>> checkpoints, but no checkpoint directory has been configured. You can
>>>> configure configure one via key 'state.checkpoints.dir'.
>>>>
>>>> Any suggestions?
>>>>
>>>> Thanks
>>
Re: state.checkpoints.dir not configured
Posted by Ufuk Celebi <uc...@apache.org>.
Could you please share the complete logs of the initial failure? What
you describe in your second email should not happen ;-) If the
JobManager cannot bind to the port it should simply die and not
complain about checkpoint configuration.
– Ufuk
On Thu, Dec 21, 2017 at 1:21 PM, Plamen Paskov
<pl...@next-stream.com> wrote:
> I inspected the log as you suggest and found that 6123 port was used by
> another process. I free the port and restarted the job manager. Now
> everything looks fine. The error message is little misleading as the real
> cause is that 6123 is already bind but it says that state.checkpoints.dir is
> not set.
>
> Thanks
>
>
>
> On 19.12.2017 17:55, Ufuk Celebi wrote:
>>
>> When the JobManager/TaskManager are starting up they log what config
>> they are loading. Look for lines like
>>
>> "Loading configuration property: {}, {}"
>>
>> Do you find the required configuration as part of these messages?
>>
>> – Ufuk
>>
>>
>> On Tue, Dec 19, 2017 at 3:45 PM, Plamen Paskov
>> <pl...@next-stream.com> wrote:
>>>
>>> Hi,
>>> I'm trying to enable externalized checkpoints like this:
>>>
>>> env.enableCheckpointing(1000);
>>> CheckpointConfig checkpointConfig = env.getCheckpointConfig();
>>>
>>> checkpointConfig.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
>>> checkpointConfig.setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
>>> env.setStateBackend(new
>>> FsStateBackend("file:///tmp/flink-checkpoints-data/", true));
>>>
>>> in flink-conf.yaml i set:
>>> state.checkpoints.dir: file:///tmp/flink-checkpoints-meta/
>>>
>>> but when i run the application i get this error:
>>> java.lang.IllegalStateException: CheckpointConfig says to persist
>>> periodic
>>> checkpoints, but no checkpoint directory has been configured. You can
>>> configure configure one via key 'state.checkpoints.dir'.
>>>
>>> Any suggestions?
>>>
>>> Thanks
>
>
Re: state.checkpoints.dir not configured
Posted by Plamen Paskov <pl...@next-stream.com>.
I inspected the log as you suggest and found that 6123 port was used by
another process. I free the port and restarted the job manager. Now
everything looks fine. The error message is little misleading as the
real cause is that 6123 is already bind but it says that
state.checkpoints.dir is not set.
Thanks
On 19.12.2017 17:55, Ufuk Celebi wrote:
> When the JobManager/TaskManager are starting up they log what config
> they are loading. Look for lines like
>
> "Loading configuration property: {}, {}"
>
> Do you find the required configuration as part of these messages?
>
> – Ufuk
>
>
> On Tue, Dec 19, 2017 at 3:45 PM, Plamen Paskov
> <pl...@next-stream.com> wrote:
>> Hi,
>> I'm trying to enable externalized checkpoints like this:
>>
>> env.enableCheckpointing(1000);
>> CheckpointConfig checkpointConfig = env.getCheckpointConfig();
>> checkpointConfig.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
>> checkpointConfig.setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
>> env.setStateBackend(new
>> FsStateBackend("file:///tmp/flink-checkpoints-data/", true));
>>
>> in flink-conf.yaml i set:
>> state.checkpoints.dir: file:///tmp/flink-checkpoints-meta/
>>
>> but when i run the application i get this error:
>> java.lang.IllegalStateException: CheckpointConfig says to persist periodic
>> checkpoints, but no checkpoint directory has been configured. You can
>> configure configure one via key 'state.checkpoints.dir'.
>>
>> Any suggestions?
>>
>> Thanks
Re: state.checkpoints.dir not configured
Posted by Ufuk Celebi <uc...@apache.org>.
When the JobManager/TaskManager are starting up they log what config
they are loading. Look for lines like
"Loading configuration property: {}, {}"
Do you find the required configuration as part of these messages?
– Ufuk
On Tue, Dec 19, 2017 at 3:45 PM, Plamen Paskov
<pl...@next-stream.com> wrote:
> Hi,
> I'm trying to enable externalized checkpoints like this:
>
> env.enableCheckpointing(1000);
> CheckpointConfig checkpointConfig = env.getCheckpointConfig();
> checkpointConfig.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
> checkpointConfig.setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
> env.setStateBackend(new
> FsStateBackend("file:///tmp/flink-checkpoints-data/", true));
>
> in flink-conf.yaml i set:
> state.checkpoints.dir: file:///tmp/flink-checkpoints-meta/
>
> but when i run the application i get this error:
> java.lang.IllegalStateException: CheckpointConfig says to persist periodic
> checkpoints, but no checkpoint directory has been configured. You can
> configure configure one via key 'state.checkpoints.dir'.
>
> Any suggestions?
>
> Thanks