You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by tr...@apache.org on 2020/02/03 12:58:22 UTC
[flink] branch release-1.10 updated (c2157a6 -> 9617d58)
This is an automated email from the ASF dual-hosted git repository.
trohrmann pushed a change to branch release-1.10
in repository https://gitbox.apache.org/repos/asf/flink.git.
from c2157a6 [FLINK-15837] Add proper NOTICE file to flink-kubernetes
new cc3f2c0 [FLINK-15806][yarn] Log recommended way how to stop a detached Yarn session cluster
new 9617d58 [FLINK-15806][doc] Update yarn documentation to include proper way of stopping a detached Yarn session
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/ops/deployment/yarn_setup.md | 6 ++++--
docs/ops/deployment/yarn_setup.zh.md | 6 ++++--
.../org/apache/flink/yarn/YarnClusterDescriptor.java | 17 +++++++++++++----
.../org/apache/flink/yarn/cli/FlinkYarnSessionCli.java | 4 +---
4 files changed, 22 insertions(+), 11 deletions(-)
[flink] 02/02: [FLINK-15806][doc] Update yarn documentation to
include proper way of stopping a detached Yarn session
Posted by tr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
trohrmann pushed a commit to branch release-1.10
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 9617d58b7f7e6a21941ad3551dc4bdcda11ca074
Author: Till Rohrmann <tr...@apache.org>
AuthorDate: Thu Jan 30 18:33:31 2020 +0100
[FLINK-15806][doc] Update yarn documentation to include proper way of stopping a detached Yarn session
This closes #10964.
---
docs/ops/deployment/yarn_setup.md | 6 ++++--
docs/ops/deployment/yarn_setup.zh.md | 6 ++++--
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/docs/ops/deployment/yarn_setup.md b/docs/ops/deployment/yarn_setup.md
index 343b55c..31f6bce 100644
--- a/docs/ops/deployment/yarn_setup.md
+++ b/docs/ops/deployment/yarn_setup.md
@@ -141,9 +141,11 @@ If you do not want to keep the Flink YARN client running all the time, it's also
The parameter for that is called `-d` or `--detached`.
In that case, the Flink YARN client will only submit Flink to the cluster and then close itself.
-Note that in this case its not possible to stop the YARN session using Flink.
-Use the YARN utilities (`yarn application -kill <appId>`) to stop the YARN session.
+In order to stop the Flink cluster gracefully use the following command: `echo "stop" | ./bin/yarn-session.sh -id <appId>`.
+
+If this should not be possible, then you can also kill Flink via YARN's web interface or via its utilities: `yarn application -kill <appId>`.
+Note, however, that killing Flink might not clean up all job artifacts and temporary files.
#### Attach to an existing Session
diff --git a/docs/ops/deployment/yarn_setup.zh.md b/docs/ops/deployment/yarn_setup.zh.md
index e31a4fe..cc69865 100644
--- a/docs/ops/deployment/yarn_setup.zh.md
+++ b/docs/ops/deployment/yarn_setup.zh.md
@@ -141,9 +141,11 @@ If you do not want to keep the Flink YARN client running all the time, it's also
The parameter for that is called `-d` or `--detached`.
In that case, the Flink YARN client will only submit Flink to the cluster and then close itself.
-Note that in this case its not possible to stop the YARN session using Flink.
-Use the YARN utilities (`yarn application -kill <appId>`) to stop the YARN session.
+In order to stop the Flink cluster gracefully use the following command: `echo "stop" | ./bin/yarn-session.sh -id <appId>`.
+
+If this should not be possible, then you can also kill Flink via YARN's web interface or via its utilities: `yarn application -kill <appId>`.
+Note, however, that killing Flink might not clean up all job artifacts and temporary files.
#### Attach to an existing Session
[flink] 01/02: [FLINK-15806][yarn] Log recommended way how to stop
a detached Yarn session cluster
Posted by tr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
trohrmann pushed a commit to branch release-1.10
in repository https://gitbox.apache.org/repos/asf/flink.git
commit cc3f2c02173fb96d75902d8479f6cd7375f85a34
Author: Till Rohrmann <tr...@apache.org>
AuthorDate: Wed Jan 29 17:51:57 2020 +0100
[FLINK-15806][yarn] Log recommended way how to stop a detached Yarn session cluster
The recommended way to shut down a detached Yarn session cluster is to reattach via
the yarn-session.sh script and to issue a stop command. This will ensure that all
job artifacts and temporary files are properly cleaned up.
---
.../org/apache/flink/yarn/YarnClusterDescriptor.java | 17 +++++++++++++----
.../org/apache/flink/yarn/cli/FlinkYarnSessionCli.java | 4 +---
2 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java b/flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
index 131685a..4a905ae 100644
--- a/flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
+++ b/flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
@@ -496,10 +496,8 @@ public class YarnClusterDescriptor implements ClusterDescriptor<ApplicationId> {
// print the application id for user to cancel themselves.
if (detached) {
- LOG.info("The Flink YARN client has been started in detached mode. In order to stop " +
- "Flink on YARN, use the following command or a YARN web interface to stop " +
- "it:\nyarn application -kill " + report.getApplicationId() + "\nPlease also note that the " +
- "temporary files of the YARN session in the home directory will not be removed.");
+ final ApplicationId yarnApplicationId = report.getApplicationId();
+ logDetachedClusterInformation(yarnApplicationId, LOG);
}
setClusterEntrypointInfoToConfig(report);
@@ -1610,5 +1608,16 @@ public class YarnClusterDescriptor implements ClusterDescriptor<ApplicationId> {
flinkConfiguration.set(YarnConfigOptions.APPLICATION_ID, ConverterUtils.toString(clusterId));
}
+
+ public static void logDetachedClusterInformation(ApplicationId yarnApplicationId, Logger logger) {
+ logger.info(
+ "The Flink YARN session cluster has been started in detached mode. In order to " +
+ "stop Flink gracefully, use the following command:\n" +
+ "$ echo \"stop\" | ./bin/yarn-session.sh -id {}\n" +
+ "If this should not be possible, then you can also kill Flink via YARN's web interface or via:\n" +
+ "$ yarn application -kill {}\n" +
+ "Note that killing Flink might not clean up all job artifacts and temporary files.",
+ yarnApplicationId, yarnApplicationId);
+ }
}
diff --git a/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java b/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java
index 4dc20c7..9955e37 100644
--- a/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java
+++ b/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java
@@ -575,9 +575,7 @@ public class FlinkYarnSessionCli extends AbstractCustomCommandLine {
}
if (!configuration.getBoolean(DeploymentOptions.ATTACHED)) {
- LOG.info("The Flink YARN client has been started in detached mode. In order to stop " +
- "Flink on YARN, use the following command or a YARN web interface to stop it:\n" +
- "yarn application -kill " + yarnApplicationId);
+ YarnClusterDescriptor.logDetachedClusterInformation(yarnApplicationId, LOG);
} else {
ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();