You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zhangminglei <gi...@git.apache.org> on 2017/07/18 07:09:44 UTC
[GitHub] flink pull request #4356: [FLINK-5486] Fix lacking of synchronization in Buc...
GitHub user zhangminglei opened a pull request:
https://github.com/apache/flink/pull/4356
[FLINK-5486] Fix lacking of synchronization in BucketingSink#handleRe…
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zhangminglei/flink flink-5486
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4356.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4356
----
commit 7399ce71c47c242566d5f6959fec2ee666929b89
Author: zhangminglei <zm...@163.com>
Date: 2017-07-18T07:07:09Z
[FLINK-5486] Fix lacking of synchronization in BucketingSink#handleRestoredBucketState
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by tedyu <gi...@git.apache.org>.
Github user tedyu commented on the issue:
https://github.com/apache/flink/pull/4356
Run QA again ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by tedyu <gi...@git.apache.org>.
Github user tedyu commented on the issue:
https://github.com/apache/flink/pull/4356
retest this please
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/4356
```
Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 91.985 sec <<< FAILURE! - in org.apache.flink.test.checkpointing.ExternalizedCheckpointITCase
testExternalizedFullRocksDBCheckpointsStandalone(org.apache.flink.test.checkpointing.ExternalizedCheckpointITCase) Time elapsed: 29.976 sec <<< ERROR!
java.io.IOException: java.lang.Exception: Failed to complete checkpoint
at org.apache.flink.runtime.testingUtils.TestingCluster.requestCheckpoint(TestingCluster.scala:399)
at org.apache.flink.test.checkpointing.ExternalizedCheckpointITCase.testExternalizedCheckpoints(ExternalizedCheckpointITCase.java:218)
at org.apache.flink.test.checkpointing.ExternalizedCheckpointITCase.testExternalizedFullRocksDBCheckpointsStandalone(ExternalizedCheckpointITCase.java:78)
Caused by: java.lang.Exception: Failed to complete checkpoint
at org.apache.flink.runtime.testingUtils.TestingJobManagerLike$$anonfun$handleTestingMessage$1$$anon$2.apply(TestingJobManagerLike.scala:375)
at org.apache.flink.runtime.testingUtils.TestingJobManagerLike$$anonfun$handleTestingMessage$1$$anon$2.apply(TestingJobManagerLike.scala:358)
at org.apache.flink.runtime.concurrent.impl.FlinkFuture$5.onComplete(FlinkFuture.java:272)
at akka.dispatch.OnComplete.internal(Future.scala:247)
at akka.dispatch.OnComplete.internal(Future.scala:245)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:175)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:172)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/4356
@tedyu I am retesting now......let us wait what will happen.
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/4356
Okay. Ted. I will trigger the QA soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by tedyu <gi...@git.apache.org>.
Github user tedyu commented on the issue:
https://github.com/apache/flink/pull/4356
lgtm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/4356
triggering now....
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/4356
OKay. Thanks , Ted. Will retest again soon enough./
---
[GitHub] flink issue #4356: [FLINK-5486] Fix lacking of synchronization in BucketingS...
Posted by GJL <gi...@git.apache.org>.
Github user GJL commented on the issue:
https://github.com/apache/flink/pull/4356
Thanks for your contribution. I have a few questions because the submitted code does not match with what the ticket currently says:
From https://issues.apache.org/jira/browse/FLINK-5486:
>The handlePendingFilesForPreviousCheckpoints() call should be enclosed inside the synchronization block. Otherwise during the processing of handlePendingFilesForPreviousCheckpoints(), some entries of the map may be cleared.
I am confused because the submitted code encloses `bucketState.pendingFilesPerCheckpoint.clear();` in a synchronized block.
@tedyu Did you run into actual production issues, or did you find the problem while reviewing the code? Can you elaborate why the additional synchronized block is needed?
---