You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2016/06/17 07:42:21 UTC
[GitHub] flink pull request #2119: [FLINK-3949] Add numSplitsProcessed (Streaming)
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/2119
[FLINK-3949] Add numSplitsProcessed (Streaming)
This PR is a small follow-up to #2090. The number of splits processed is now also measured for the Streaming API. This change was excluded from the original PR due to concurrent changes to the way InputFormat are handled.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 3949+
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2119.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2119
----
commit 4defc5bc8fa29e3d99117fdd577365ce678918d7
Author: zentol <ch...@apache.org>
Date: 2016-06-17T07:40:01Z
[FLINK-3949] Add numSplitsProcessed (Streaming)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2119: [FLINK-3949] Add numSplitsProcessed (Streaming)
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2119#discussion_r67479777
--- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.java ---
@@ -292,6 +294,7 @@ public void run() {
// close and prepare for the next iteration
this.format.close();
this.currentSplit = null;
+ completedSplitsCounter.inc();
--- End diff --
Should we move the increment out of the `finally` clause to avoid counting a failed split as processed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2119: [FLINK-3949] Add numSplitsProcessed (Streaming)
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2119#discussion_r67480983
--- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.java ---
@@ -292,6 +294,7 @@ public void run() {
// close and prepare for the next iteration
this.format.close();
this.currentSplit = null;
+ completedSplitsCounter.inc();
--- End diff --
jup
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2119: [FLINK-3949] Add numSplitsProcessed (Streaming)
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2119#discussion_r67481133
--- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.java ---
@@ -292,6 +294,7 @@ public void run() {
// close and prepare for the next iteration
this.format.close();
this.currentSplit = null;
+ completedSplitsCounter.inc();
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2119: [FLINK-3949] Add numSplitsProcessed (Streaming)
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/2119
merging
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2119: [FLINK-3949] Add numSplitsProcessed (Streaming)
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/2119
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2119: [FLINK-3949] Add numSplitsProcessed (Streaming)
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/2119
Thanks for the update. +1 to merge
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---