You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Rui Fan (Jira)" <ji...@apache.org> on 2023/12/25 07:22:00 UTC
[jira] [Updated] (FLINK-33935) Improve the default value doc and logic for some state backend and checkpoint related options
[ https://issues.apache.org/jira/browse/FLINK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rui Fan updated FLINK-33935:
----------------------------
Description:
Some state backend and checkpoint related options don't set the default value directly, but but they implement default value based on code. Such as:
* execution.checkpointing.tolerable-failed-checkpoints
** [https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/#execution-checkpointing-tolerable-failed-checkpoints]
* state.backend.type
** [https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/#state-backend-type]
h2. Option1
execution.checkpointing.tolerable-failed-checkpoints doesn't have default value, but CheckpointConfig#getTolerableCheckpointFailureNumber calls {color:#9876aa}configuration{color}.getOptional(ExecutionCheckpointingOptions.{color:#9876aa}TOLERABLE_FAILURE_NUMBER{color}).orElse({color:#6897bb}0{color}).
It means the 0 is default value of execution.checkpointing.tolerable-failed-checkpoints.
h2. Option2
state.backend.type does't have default value, but StateBackendLoader#loadFromApplicationOrConfigOrDefaultInternal calls
loadStateBackendFromConfig(config{color:#cc7832}, {color}classLoader{color:#cc7832}, {color}logger). When the return value is null, Flink will consider the hashmap as the default state backend.
I checked all callers of StateBackendLoader#loadStateBackendFromConfig, if we change the default value of state.backend.type to hashmap. All of them work well.
h2. Why set the default value directly is fine?
From user side, it's clearer.
From flink developers or maintainers side, it's easy to maintain.
was:
Some state backend and checkpoint related options don't set the default value directly, but but they implement default value based on code. Such as:
* execution.checkpointing.tolerable-failed-checkpoints
** [https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/#execution-checkpointing-tolerable-failed-checkpoints]
* state.backend.type
** [https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/#state-backend-type]
h2. Option1
execution.checkpointing.tolerable-failed-checkpoints doesn't have default value, but CheckpointConfig#getTolerableCheckpointFailureNumber calls {color:#9876aa}configuration{color}.getOptional(ExecutionCheckpointingOptions.{color:#9876aa}TOLERABLE_FAILURE_NUMBER{color}).orElse({color:#6897bb}0{color}).
It means the 0 is default value of execution.checkpointing.tolerable-failed-checkpoints.
h2. Option2
state.backend.type does't have default value, but StateBackendLoader#loadFromApplicationOrConfigOrDefaultInternal calls
loadStateBackendFromConfig(config{color:#cc7832}, {color}classLoader{color:#cc7832}, {color}logger). When the return value is null, Flink will consider the hashmap as the default state backend.
I checked all callers of StateBackendLoader#loadStateBackendFromConfig, if we change the default value of state.backend.type to hashmap. All of them work well.
> Improve the default value doc and logic for some state backend and checkpoint related options
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-33935
> URL: https://issues.apache.org/jira/browse/FLINK-33935
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing, Runtime / State Backends
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Major
> Fix For: 1.19.0
>
>
> Some state backend and checkpoint related options don't set the default value directly, but but they implement default value based on code. Such as:
> * execution.checkpointing.tolerable-failed-checkpoints
> ** [https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/#execution-checkpointing-tolerable-failed-checkpoints]
> * state.backend.type
> ** [https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/#state-backend-type]
> h2. Option1
> execution.checkpointing.tolerable-failed-checkpoints doesn't have default value, but CheckpointConfig#getTolerableCheckpointFailureNumber calls {color:#9876aa}configuration{color}.getOptional(ExecutionCheckpointingOptions.{color:#9876aa}TOLERABLE_FAILURE_NUMBER{color}).orElse({color:#6897bb}0{color}).
> It means the 0 is default value of execution.checkpointing.tolerable-failed-checkpoints.
> h2. Option2
> state.backend.type does't have default value, but StateBackendLoader#loadFromApplicationOrConfigOrDefaultInternal calls
> loadStateBackendFromConfig(config{color:#cc7832}, {color}classLoader{color:#cc7832}, {color}logger). When the return value is null, Flink will consider the hashmap as the default state backend.
> I checked all callers of StateBackendLoader#loadStateBackendFromConfig, if we change the default value of state.backend.type to hashmap. All of them work well.
> h2. Why set the default value directly is fine?
> From user side, it's clearer.
> From flink developers or maintainers side, it's easy to maintain.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)