You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Stephan Ewen (JIRA)" <ji...@apache.org> on 2018/04/17 17:48:00 UTC
[jira] [Commented] (FLINK-9196) YARN: Flink binaries are not
deleted from HDFS after cluster shutdown
[ https://issues.apache.org/jira/browse/FLINK-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441248#comment-16441248 ]
Stephan Ewen commented on FLINK-9196:
-------------------------------------
I think this may explain parts of the error described here: https://lists.apache.org/thread.html/73b652d2f4ba1167ff8166dc8260cf3458d5555f562c630f8f891eb3@%3Cuser.flink.apache.org%3E
> YARN: Flink binaries are not deleted from HDFS after cluster shutdown
> ---------------------------------------------------------------------
>
> Key: FLINK-9196
> URL: https://issues.apache.org/jira/browse/FLINK-9196
> Project: Flink
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.5.0
> Reporter: Gary Yao
> Priority: Blocker
> Labels: flip-6
>
> When deploying on YARN in flip6 mode, the Flink binaries are not deleted from HDFS after the cluster shuts down.
> *Steps to reproduce*
> # Submit job in YARN job mode, non-detached:
> {noformat} HADOOP_CLASSPATH=`hadoop classpath` bin/flink run -m yarn-cluster -yjm 2048 -ytm 2048 -yn 2 ./examples/streaming/WordCount.jar {noformat}
> # Check contents of {{/user/hadoop/.flink/<application_id>}} on HDFS after job is finished:
> {noformat}
> [hadoop@ip-172-31-43-78 flink-1.5.0]$ hdfs dfs -ls /user/hadoop/.flink/application_1523966184826_0016
> Found 6 items
> -rw-r--r-- 1 hadoop hadoop 583 2018-04-17 14:54 /user/hadoop/.flink/application_1523966184826_0016/90cf5b3a-039e-4d52-8266-4e9563d74827-taskmanager-conf.yaml
> -rw-r--r-- 1 hadoop hadoop 332 2018-04-17 14:54 /user/hadoop/.flink/application_1523966184826_0016/application_1523966184826_0016-flink-conf.yaml3818971235442577934.tmp
> -rw-r--r-- 1 hadoop hadoop 89779342 2018-04-02 17:08 /user/hadoop/.flink/application_1523966184826_0016/flink-dist_2.11-1.5.0.jar
> drwxrwxrwx - hadoop hadoop 0 2018-04-17 14:54 /user/hadoop/.flink/application_1523966184826_0016/lib
> -rw-r--r-- 1 hadoop hadoop 1939 2018-04-02 15:37 /user/hadoop/.flink/application_1523966184826_0016/log4j.properties
> -rw-r--r-- 1 hadoop hadoop 2331 2018-04-02 15:37 /user/hadoop/.flink/application_1523966184826_0016/logback.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)