[jira] [Created] (FLINK-16864) Add idle metrics for Task

["Wenlong Lyu (Jira)" <ji...@apache.org>]

Wenlong Lyu created FLINK-16864:
-----------------------------------

             Summary: Add idle metrics for Task
                 Key: FLINK-16864
                 URL: https://issues.apache.org/jira/browse/FLINK-16864
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Task
            Reporter: Wenlong Lyu


Currently there is no metric for user to measure how busy a task is concretely, which is important for user to decide how to tune a job.

We would like to propose adding an IdleTime which measure idle time of a task including the time cost for mail processor to wait for new mail and the time cost in record writer to waiting a new buffer. 

With the idle time:
1. when a job can not catch up with the speed of data generating, the vertex which idle time is near to zero is the bottle neck of the job.
2. when a job is not busy, idle time  can be used to guide user how much he can scale down the job.

In addition, measuring idle time can have little impaction on the performance of the job, because when a task is busy, we don't touch the code to measure wait-time in mailbox.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)