You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Ting Sun (Jira)" <ji...@apache.org> on 2021/01/20 12:06:00 UTC
[jira] [Issue Comment Deleted] (FLINK-18019) The configuration
specified in TableConfig may not take effect in certain cases
[ https://issues.apache.org/jira/browse/FLINK-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ting Sun updated FLINK-18019:
-----------------------------
Comment: was deleted
(was: [~dian.fu] [~dwysakowicz] Sorry for the late reply. I have reproduced the issue. I'm new to Flink, so could you please give me some insights into the issue? Any information will be appreciated.)
> The configuration specified in TableConfig may not take effect in certain cases
> -------------------------------------------------------------------------------
>
> Key: FLINK-18019
> URL: https://issues.apache.org/jira/browse/FLINK-18019
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Legacy Planner, Table SQL / Planner
> Affects Versions: 1.10.0, 1.11.0
> Reporter: Dian Fu
> Priority: Major
>
> Currently If the following configuration is configured in flink-conf.yaml:
> {code:java}
> state.backend: rocksdb
> state.checkpoints.dir: file:///tmp/flink-checkpoints
> {code}
> and the following configuration is configured via TableConfig:
> {code:java}
> tableConfig.getConfiguration().setString("state.backend.rocksdb.memory.fixed-per-slot", "200MB")
> tableConfig.getConfiguration().setString("taskmanager.memory.task.off-heap.size", "200MB")
> {code}
> Then users submit the job via CliFrontend, the configuration set via TableConfig will not take effect.
> Intuitively, it should be that user specified configuration via TableConfig(has higher priority) and the configuration specified via flink-conf.yaml together determines the configuration of a job. However, it doesn't hold in all cases.
> The root cause is that only the configuration specified in TableConfig in passed to *StreamExecutionEnvironment* during translate to plan. For the above case, as *state.backend* is not specified in TableConfig and so the configuration *state.backend.rocksdb.memory.fixed-per-slot* will not take effect. Please note that in above example, the state backend actually used will be RocksDB without the configurationĀ *state.backend.rocksdb.memory.fixed-per-slot* andĀ *taskmanager.memory.task.off-heap.size.*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)