You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by uce <gi...@git.apache.org> on 2015/08/24 14:53:02 UTC
[GitHub] flink pull request: [FLINK-2089] [runtime] Fix illegal state in Re...
GitHub user uce opened a pull request:
https://github.com/apache/flink/pull/1050
[FLINK-2089] [runtime] Fix illegal state in RecordWriter after partition write failure
I'm waiting for feedback from a user whether this fixes FLINK-2089, but this PR definitely addresses a problem.
Record writers have a `clearBuffers` method, which is called by the task code in a finally block at the end of `invoke` (see `RegularPactTask` for example). This call clears the buffers of the record serializers.
The following illegal state can arise: a buffer has been published to a partition, but the serializers still hold a reference to it. When a serializer tries to clear its current buffer, it might have already been recycled (because it was published to the partition). This will currently happen if there was an Exception during writing the buffer to the partition.
This PR replaces the write-and-clear calls with a try-catch-finally block and tests the expected behaviour in a new test. The removed tests are superseded by this new test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/uce/flink illegal-2089-master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1050.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1050
----
commit 88ca58b5e3e78354ca1cffee4e11b48011333c6b
Author: Ufuk Celebi <uc...@apache.org>
Date: 2015-08-19T14:11:13Z
[FLINK-2089] [runtime] Fix illegal state in RecordWriter after partition write failure
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2089] [runtime] Fix illegal state in Re...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the pull request:
https://github.com/apache/flink/pull/1050#issuecomment-134256139
The failed Travis build is due to FLINK-2564 (see #1047 for the fix). The other builds pass.
**Note**: This needs to be merged to `release-0.9` as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2089] [runtime] Fix illegal state in Re...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1050#discussion_r37861035
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java ---
@@ -20,7 +20,7 @@
import org.apache.flink.core.io.IOReadableWritable;
import org.apache.flink.runtime.accumulators.AccumulatorRegistry;
-import org.apache.flink.runtime.event.AbstractEvent;
+import org.apache.flink.runtime.event.AbstractEvent;
--- End diff --
Whitespace should be removed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2089] [runtime] Fix illegal state in Re...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the pull request:
https://github.com/apache/flink/pull/1050#issuecomment-134676836
I will address the comment and merge this for 0.10 and 0.9.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2089] [runtime] Fix illegal state in Re...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/1050#issuecomment-134634885
LGTM +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2089] [runtime] Fix illegal state in Re...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on a diff in the pull request:
https://github.com/apache/flink/pull/1050#discussion_r37865266
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java ---
@@ -20,7 +20,7 @@
import org.apache.flink.core.io.IOReadableWritable;
import org.apache.flink.runtime.accumulators.AccumulatorRegistry;
-import org.apache.flink.runtime.event.AbstractEvent;
+import org.apache.flink.runtime.event.AbstractEvent;
--- End diff --
Good catch. I'm wondering how that happened though as I don't import stuff manually... :dart:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2089] [runtime] Fix illegal state in Re...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/1050
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---