[jira] [Commented] (FLINK-3302) Save Taskmanager Metrics per Job

["Greg Hogan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123589#comment-15123589 ] 

Greg Hogan commented on FLINK-3302:
-----------------------------------

Hi, Pieter! I had filed FLINK-3263 about a week ago with a similar idea, the only difference being whether to log on the JobManager or TaskManager.

I see advantages of logging on the TaskManager as:
- can decouple the logging interval from the heartbeat interval
- writes logs across multiple TaskManagers rather than duplicating logging across potentially many JobManagers
- logging is guaranteed whereas heartbeats may be lost

I see the advantage of logging on the JobManager as generally simplifying the configuration (potential reuse of the heartbeat interval) and post-processing (no need to fetch and parse multiple logs).

> Save Taskmanager Metrics per Job
> --------------------------------
>
>                 Key: FLINK-3302
>                 URL: https://issues.apache.org/jira/browse/FLINK-3302
>             Project: Flink
>          Issue Type: Wish
>          Components: JobManager, Webfrontend
>            Reporter: Pieter Hameete
>            Priority: Minor
>
> As followup to my email in the user mailing list:
> It would be nice if there is a configuration option to have the JobManager save the received metrics for each of the TaskManagers per separate job. This information can be useful for later analysis of job performance.
> The Metrics are currently already received by the JobManager and shown/updated in the Web UI every few seconds. It should not be very difficult to write these timestamped updates to disk and group per job.
> This is something I could pick up myself in a couple of months.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)