[jira] [Commented] (FLINK-24555) Incorrectly put empty list to flink configuration

["Junfan Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-24555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429095#comment-17429095 ] 

Junfan Zhang commented on FLINK-24555:
--------------------------------------

[~jark] Could you help review it? Thanks

> Incorrectly put empty list to flink configuration
> -------------------------------------------------
>
>                 Key: FLINK-24555
>                 URL: https://issues.apache.org/jira/browse/FLINK-24555
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Junfan Zhang
>            Priority: Major
>
> h2. Why
> As I found some warn logs in our production flink jobs, like as follows(the detail logs attached at the end)
> {code:java}
> 2021-10-14 17:46:41 WARN  - [main] - org.apache.flink.configuration.GlobalConfiguration(186) - Error while trying to split key and value in configuration file /data8/yarn/local/usercache/pizza/appcache/application_1633896099002_1043646/container_e429_1633896099002_1043646_01_000001/flink-conf.yaml:11: "pipeline.classpaths: "
> {code}
> I dig the flink code and found it put the empty list into configuration by {{ConfigUtils.encodeCollectionToConfig}}.
> h2. How
> So it's better to ignore the empty collection to put in the configuration.
> Because these configuration will be written to the flink-conf.yaml and Flink jobmanager will read but fail to read the corresponding key's value as the warn log described.
> h2. Appendix
> {code:java}
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: internal.jobgraph-path, job.graph
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: restart-strategy.failure-rate.max-failures-per-interval, 3
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: jobmanager.execution.failover-strategy, region
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: high-availability.cluster-id, application_1633896099002_1043646
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: jobmanager.rpc.address, localhost
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: execution.runtime-mode, AUTOMATIC
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: security.kerberos.fetch.delegation-token, false
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: execution.savepoint.ignore-unclaimed-state, false
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: parallelism.default, 2
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: taskmanager.numberOfTaskSlots, 1
> 2021-10-14 17:46:41 WARN  - [main] - org.apache.flink.configuration.GlobalConfiguration(186) - Error while trying to split key and value in configuration file /data8/yarn/local/usercache/pizza/appcache/application_1633896099002_1043646/container_e429_1633896099002_1043646_01_000001/flink-conf.yaml:11: "pipeline.classpaths: "
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: restart-strategy.failure-rate.failure-rate-interval, 1 d
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: yarn.application.name, app_StreamEngine_Prod_jiandan_beat_rec_test_all
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: yarn.application.queue, talos.job_streaming
> 2021-10-14 17:46:41 INFO  - [main] - org.apache.flink.configuration.GlobalConfiguration(213) - Loading configuration property: taskmanager.memory.process.size, 1728m
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)