You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Shangwen Tang (Jira)" <ji...@apache.org> on 2020/05/22 17:15:00 UTC
[jira] [Comment Edited] (FLINK-17891) FlinkYarnSessionCli sets
wrong execution.target type
[ https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114246#comment-17114246 ]
Shangwen Tang edited comment on FLINK-17891 at 5/22/20, 5:14 PM:
-----------------------------------------------------------------
My idea is that if we start the flink session job with FlinkYarnSessionCli, we should set the *execution.target* to be of yarn-session type, not yarn-per-job. and the problem is in this line
{{}}{{{noformat}
effectiveConfiguration.setString(DeploymentOptions.TARGET, YarnJobClusterExecutor.NAME);}}{{{noformat}}}
{code:java}
// FlinkYarnSessionCli.java
@Override
public Configuration applyCommandLineOptionsToConfiguration(CommandLine commandLine) throws FlinkException {
// we ignore the addressOption because it can only contain "yarn-cluster"
final Configuration effectiveConfiguration = new Configuration(configuration);
applyDescriptorOptionToConfig(commandLine, effectiveConfiguration);
final ApplicationId applicationId = getApplicationId(commandLine);
if (applicationId != null) {
final String zooKeeperNamespace;
if (commandLine.hasOption(zookeeperNamespace.getOpt())){
zooKeeperNamespace = commandLine.getOptionValue(zookeeperNamespace.getOpt());
} else {
zooKeeperNamespace = effectiveConfiguration.getString(HA_CLUSTER_ID, applicationId.toString());
}
effectiveConfiguration.setString(HA_CLUSTER_ID, zooKeeperNamespace);
effectiveConfiguration.setString(YarnConfigOptions.APPLICATION_ID, ConverterUtils.toString(applicationId));
effectiveConfiguration.setString(DeploymentOptions.TARGET, YarnSessionClusterExecutor.NAME);
} else {
effectiveConfiguration.setString(DeploymentOptions.TARGET, YarnJobClusterExecutor.NAME);
}
...
}{code}
was (Author: tangshangwen):
My idea is that if we start the flink session job with FlinkYarnSessionCli, we should set the *execution.target* to be of yarn-session type, not yarn-per-job. and that's where the problem comes in `effectiveConfiguration.setString(DeploymentOptions.TARGET, YarnJobClusterExecutor.NAME);`
{code:java}
// FlinkYarnSessionCli.java
@Override
public Configuration applyCommandLineOptionsToConfiguration(CommandLine commandLine) throws FlinkException {
// we ignore the addressOption because it can only contain "yarn-cluster"
final Configuration effectiveConfiguration = new Configuration(configuration);
applyDescriptorOptionToConfig(commandLine, effectiveConfiguration);
final ApplicationId applicationId = getApplicationId(commandLine);
if (applicationId != null) {
final String zooKeeperNamespace;
if (commandLine.hasOption(zookeeperNamespace.getOpt())){
zooKeeperNamespace = commandLine.getOptionValue(zookeeperNamespace.getOpt());
} else {
zooKeeperNamespace = effectiveConfiguration.getString(HA_CLUSTER_ID, applicationId.toString());
}
effectiveConfiguration.setString(HA_CLUSTER_ID, zooKeeperNamespace);
effectiveConfiguration.setString(YarnConfigOptions.APPLICATION_ID, ConverterUtils.toString(applicationId));
effectiveConfiguration.setString(DeploymentOptions.TARGET, YarnSessionClusterExecutor.NAME);
} else {
effectiveConfiguration.setString(DeploymentOptions.TARGET, YarnJobClusterExecutor.NAME);
}
...
}{code}
> FlinkYarnSessionCli sets wrong execution.target type
> -----------------------------------------------------
>
> Key: FLINK-17891
> URL: https://issues.apache.org/jira/browse/FLINK-17891
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.11.0
> Reporter: Shangwen Tang
> Priority: Major
> Attachments: image-2020-05-23-00-59-32-702.png, image-2020-05-23-01-00-19-549.png
>
>
> I submitted a flink session job at the local YARN cluster, and I found that the *execution.target* is of the wrong type, which should be of yarn-session type
> !image-2020-05-23-00-59-32-702.png|width=545,height=75!
> !image-2020-05-23-01-00-19-549.png|width=544,height=94!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)