You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2018/06/06 08:00:24 UTC
[GitHub] flink pull request #6126: [FLINK-9530][metrics] Fix numRecords task metric f...
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/6126
[FLINK-9530][metrics] Fix numRecords task metric for chains
## What is the purpose of the change
This PR fixes the task re-use of operator `numRecordsIn/Out` metrics. Tasks were re-using both metrics for all operators in a chain, such that `numRecordsIn` for a task was the sum of `numRecordsIn` over all operators.
This was caused due to a copy&paste error in a previous refactoring. The removed lines in `ChainingOutput` are entirely unnecessary as the re-use is completely handled by `AbstractStreamOperator#setup`.
## Brief change log
* remove faulty code from `ChainingOutput`
* add tests
## Verifying this change
* see added tests in `OneInputStreamTaskTest` and `TwoInputStreamTaskTest`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 9530
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6126.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6126
----
commit cb47df38e72caef557749baee0d8a21ad29ac1ab
Author: zentol <ch...@...>
Date: 2018-06-05T13:01:10Z
[FLINK-9530][metrics] Fix numRecords task metric for chains
----
---
[GitHub] flink pull request #6126: [FLINK-9530][metrics] Fix numRecords task metric f...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/6126
---
[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/6126
yes, the `numRecordsIn` metric for a `ChainingOutput` is irrelevant for the task metrics and thus can be removed without affecting the test.
---
[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/6126
No we can't remove it as it is still used for the operator.
---
[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/6126
This is the very counter that, for a chain A -> B, counts the outgoing records of A and incoming records of B.
---
[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...
Posted by kl0u <gi...@git.apache.org>.
Github user kl0u commented on the issue:
https://github.com/apache/flink/pull/6126
I got this, it is just that I am wondering how useful this is, given that the operators/functions are chained.
---
[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...
Posted by kl0u <gi...@git.apache.org>.
Github user kl0u commented on the issue:
https://github.com/apache/flink/pull/6126
If this is correct, then I suppose we can remove it altogether so that the code is also cleaner, right?
---
[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/6126
This gives you a consistent view about operators regardless of whether they are chained or not.
I don't quite understand how you could consider this metric not being useful.
---
[GitHub] flink issue #6126: [FLINK-9530][metrics] Fix numRecords task metric for chai...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/6126
merging.
---