[jira] [Comment Edited] (FLINK-16864) Add idle metrics for Task

["Piotr Nowojski (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-16864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078339#comment-17078339 ] 

Piotr Nowojski edited comment on FLINK-16864 at 4/8/20, 2:32 PM:
-----------------------------------------------------------------

I had to revert this change because FLINK-17053 and FLINK-17054.

Re FLINK-17053 [~wenlong.lwl]. The test has to be rewritten to avoid mockito usage. For example instead of verifying how many times a method has been called on a mock:
{code}
BufferProvider bufferProvider = mock(BufferProvider.class);
{code}
Use {{TestPooledBufferProvider}}

reverted d69d3c0589 on master.


was (Author: pnowojski):
I had to revert this change because FLINK-17053 and FLINK-17054.

reverted d69d3c0589 on master.

> Add idle metrics for Task
> -------------------------
>
>                 Key: FLINK-16864
>                 URL: https://issues.apache.org/jira/browse/FLINK-16864
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Metrics, Runtime / Task
>            Reporter: Wenlong Lyu
>            Assignee: Wenlong Lyu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.11.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently there is no metric for user to measure how busy a task is concretely, which is important for user to decide how to tune a job.
> We would like to propose adding an IdleTime which measure idle time of a task including the time cost for mail processor to wait for new mail and the time cost in record writer to waiting a new buffer. 
> With the idle time:
> 1. when a job can not catch up with the speed of data generating, the vertex which idle time is near to zero is the bottle neck of the job.
> 2. when a job is not busy, idle time  can be used to guide user how much he can scale down the job.
> In addition, measuring idle time can have little impaction on the performance of the job, because when a task is busy, we don't touch the code to measure wait-time in mailbox.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)