You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2018/06/06 08:00:24 UTC

[GitHub] flink pull request #6126: [FLINK-9530][metrics] Fix numRecords task metric f...

GitHub user zentol opened a pull request:

    https://github.com/apache/flink/pull/6126

    [FLINK-9530][metrics] Fix numRecords task metric for chains

    ## What is the purpose of the change
    
    This PR fixes the task re-use of operator `numRecordsIn/Out` metrics. Tasks were re-using both metrics for all operators in a chain, such that `numRecordsIn` for a task was the sum of `numRecordsIn` over all operators.
    
    This was caused due to a copy&paste error in a previous refactoring. The removed lines in `ChainingOutput` are entirely unnecessary as the re-use is completely handled by `AbstractStreamOperator#setup`.
    
    ## Brief change log
    
    * remove faulty code from `ChainingOutput`
    * add tests
    
    ## Verifying this change
    
    * see added tests in `OneInputStreamTaskTest` and `TwoInputStreamTaskTest`


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 9530

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/6126.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #6126
    
----
commit cb47df38e72caef557749baee0d8a21ad29ac1ab
Author: zentol <ch...@...>
Date:   2018-06-05T13:01:10Z

    [FLINK-9530][metrics] Fix numRecords task metric for chains

----


---

[GitHub] flink pull request #6126: [FLINK-9530][metrics] Fix numRecords task metric f...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/6126


---

[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/6126
  
    yes, the `numRecordsIn` metric for a `ChainingOutput` is irrelevant for the task metrics and thus can be removed without affecting the test.


---

[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/6126
  
    No we can't remove it as it is still used for the operator.


---

[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/6126
  
    This is the very counter that, for a chain A -> B, counts the outgoing records of A and incoming records of B.


---

[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...

Posted by kl0u <gi...@git.apache.org>.
Github user kl0u commented on the issue:

    https://github.com/apache/flink/pull/6126
  
    I got this, it is just that I am wondering how useful this is, given that the operators/functions are chained. 


---

[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...

Posted by kl0u <gi...@git.apache.org>.
Github user kl0u commented on the issue:

    https://github.com/apache/flink/pull/6126
  
    If this is correct, then I suppose we can remove it altogether so that the code is also cleaner, right?


---

[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/6126
  
    This gives you a consistent view about operators regardless of whether they are chained or not.
    
    I don't quite understand how you could consider this metric not being useful.


---

[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/6126
  
    merging.


---