[jira] [Comment Edited] (FLINK-34114) Parse error while $internal.application.program-args contains '#' in Yarn/K8s Application Mode

["jonasjc (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-34114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17807504#comment-17807504 ] 

jonasjc edited comment on FLINK-34114 at 1/17/24 2:14 AM:
----------------------------------------------------------

[~MartijnVisser] Hi,i have already fix this problem and valided in my company,if the plan has no problem ,assign to me please.


was (Author: JIRAUSER295075):
Hi,i have already fix this problem and valided in my company,if the plan has no problem ,assign to me please.

> Parse error while $internal.application.program-args contains '#' in Yarn/K8s Application Mode
> ----------------------------------------------------------------------------------------------
>
>                 Key: FLINK-34114
>                 URL: https://issues.apache.org/jira/browse/FLINK-34114
>             Project: Flink
>          Issue Type: Bug
>          Components: Deployment / Kubernetes, Deployment / YARN
>    Affects Versions: 1.18.0
>            Reporter: jonasjc
>            Priority: Major
>
> When run job by K8s or Yarn Application Mode use org.apache.flink.configuration.GlobalConfiguration#loadYAMLResource method parse config。
>  
> {code:java}
> private static Configuration loadYAMLResource(File file) {
>     final Configuration config = new Configuration();
>     try (BufferedReader reader =
>             new BufferedReader(new InputStreamReader(new FileInputStream(file)))) {
>         String line;
>         int lineNo = 0;
>         while ((line = reader.readLine()) != null) {
>             lineNo++;
>             // 1. check for comments
>             String[] comments = line.split("#", 2);
>             String conf = comments[0].trim();
>             // 2. get key and value
>             if (conf.length() > 0) {
>                 String[] kv = conf.split(": ", 2);
>                 // skip line with no valid key-value pair
>                 if (kv.length == 1) {
>                     LOG.warn(
>                             "Error while trying to split key and value in configuration file "
>                                     + file
>                                     + ":"
>                                     + lineNo
>                                     + ": Line is not a key-value pair (missing space after ':'?)");
>                     continue;
>                 }
>                 String key = kv[0].trim();
>                 String value = kv[1].trim();
>                 // sanity check
>                 if (key.length() == 0 || value.length() == 0) {
>                     LOG.warn(
>                             "Error after splitting key and value in configuration file "
>                                     + file
>                                     + ":"
>                                     + lineNo
>                                     + ": Key or value was empty");
>                     continue;
>                 }
>                 config.setString(key, value);
>             }
>         }
>     } catch (IOException e) {
>         throw new RuntimeException("Error parsing YAML configuration.", e);
>     }
>     return config;
> } {code}
> if config value contains '#' like
>  
> {code:java}
> $internal.application.program-args: '{test:test#jsonstring}'{code}
>  
>  
> the following code is not correct
> {code:java}
> line.split("#", 2) {code}
> To fix this,i think we should import snakeyaml to fix complex situation



--
This message was sent by Atlassian Jira
(v8.20.10#820010)