[jira] [Commented] (FLINK-15449) Retain lost task managers on Flink UI

["Chesnay Schepler (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186411#comment-17186411 ] 

Chesnay Schepler commented on FLINK-15449:
------------------------------------------

Then we'd be adding another REST endpoint that does the same thing as an existing one, the UI would have to work for both cases (somehow), and we would have to cache task executor logs on the JobMaster, which we of course would also have to clean up.

Furthermore, I suppose you would also want these logs to be retained if the JM crashed.

I consider this to be out of Flinks scope, and would recommend using either deployment-specific options (e.g., access logs via kubectl in the case of kubernetes) or setup a dedicated log system.

> Retain lost task managers on Flink UI
> -------------------------------------
>
>                 Key: FLINK-15449
>                 URL: https://issues.apache.org/jira/browse/FLINK-15449
>             Project: Flink
>          Issue Type: Improvement
>          Components: Deployment / YARN
>    Affects Versions: 1.9.1
>            Reporter: Victor Wong
>            Priority: Major
>
> With Flink on Yarn, sometimes our TaskManager was killed because of OOM or heartbeat timeout or whatever reasons, it's not convenient to check out the logs of the lost TaskManger.
> Can we retain the lost task managers on Flink UI, and provide the log service through Yarn (we can redirect the URL of log/stdout to Yarn container log/stdout)?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)