[jira] [Created] (FLINK-16305) FlinkYarnSessionClI ignores target executor and uses yarn-session if YARN properties file is present

["Daniel Laszlo Magyar (Jira)" <ji...@apache.org>]

Daniel Laszlo Magyar created FLINK-16305:
--------------------------------------------

             Summary: FlinkYarnSessionClI ignores target executor and uses yarn-session if YARN properties file is present
                 Key: FLINK-16305
                 URL: https://issues.apache.org/jira/browse/FLINK-16305
             Project: Flink
          Issue Type: Bug
    Affects Versions: 1.10.0
            Reporter: Daniel Laszlo Magyar


The presence of the hidden YARN property file (which contains the last started YARN session’s application id), causes the cli to ignore the {{execution.target}} property set in the {{conf/flink-conf.yaml}} configuration file, which leads to unexpected behaviour at the time of job submission via cli, e.g. when using {{flink run}} or SQL client.
 The code that ignores the execution target if the YARN application id is set in the hidden property file is at [https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L337-L351].
 
Reproduction steps:
 • start flink yarn session via {{./bin/yarn-session.sh -d}}, this writes the application id to {{/tmp/.yarn-properties-root}}
 • set {{execution.target: yarn-per-job}} in {{/etc/flink/conf/flink-conf.yaml}}
 • enable debug logging
 • run a flink job e.g. {{flink run -d -p 2 examples/streaming/WordCount.jar --input README.txt}}
 • the logs below show that even though the {{execution.target}} property is read properly, {{FlinkYarnSessionCli}} is chosen and the execution.target is reset to yarn-session
{code:java}
20/02/26 12:14:24 INFO configuration.GlobalConfiguration: Loading configuration property: execution.target, yarn-per-job
...
20/02/26 12:14:24 INFO cli.FlinkYarnSessionCli: Found Yarn properties file under /tmp/.yarn-properties-root.
20/02/26 12:14:24 DEBUG fs.FileSystem: Loading extension file systems via services
20/02/26 12:14:24 DEBUG cli.CliFrontend: Custom commandlines: [org.apache.flink.yarn.cli.FlinkYarnSessionCli@43df23d3, org.apache.flink.client.cli.ExecutorCLI@6d60fe40, org.apache.flink.client.cli.DefaultCLI@792b749c]
20/02/26 12:14:24 DEBUG cli.CliFrontend: Checking custom commandline org.apache.flink.yarn.cli.FlinkYarnSessionCli@43df23d3, isActive: true
...
20/02/26 12:14:25 DEBUG cli.CliFrontend: Effective executor configuration: {...execution.target=yarn-session, ....}
20/02/26 12:14:25 INFO client.ClientUtils: Starting program (detached: true)

{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)