You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2018/01/24 12:02:16 UTC
[GitHub] flink pull request #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file...
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/5351
[FLINK-6623][Blob] BlobServer#putBuffer moves file after stream is cl…
## What is the purpose of the change
In BlobServer#putBuffer we were moving a file while we still had an open FileOutputStream for it, causing the `BlobCacheSuccessTest` to categorically fail on Windows.
The PR splits the existing try-with-resources block into 2 blocks: one for writing the data to the temporary file, and another one for moving the file to the storage directory.
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 6623
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5351.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5351
----
commit 196efd73b5af16c4e7bc51ebe16cb886342c0d80
Author: zentol <ch...@...>
Date: 2018-01-24T11:34:17Z
[FLINK-6623][Blob] BlobServer#putBuffer moves file after stream is closed
----
---
[GitHub] flink issue #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file after ...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/5351
Cool, do you want to commit both changes (this PR and the referenced branch) together?
---
[GitHub] flink issue #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file after ...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5351
Note that the tests may still sporadically fail due to FLINK-5659.
---
[GitHub] flink issue #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file after ...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5351
So the `FIleBaseUtilsTest` ran without a hitch 3000 times in a row, which is _really_ nice.
---
[GitHub] flink issue #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file after ...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5351
Taking a look at the moment. I tried it with the `BlobCacheSuccessTest` and get about 30-50% less failures, but this could also be due to some leak in the BlobServer we haven't found yet.
I'll try it with the `FileUtilsTest`, that one should be isolated from other components.
---
[GitHub] flink issue #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file after ...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/5351
@zentol I revived and rebased an old branch with an attempt to fix the concurrency issue of file deletes on Windows: https://github.com/StephanEwen/incubator-flink/commit/0504a66b82ca2afe9a40457413585ecab6d6cc33
Can you check this out, see if it works, and commit is it is helpful?
---
[GitHub] flink pull request #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5351
---
[GitHub] flink issue #5351: [FLINK-6623][Blob] BlobServer#putBuffer moves file after ...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5351
yes I'll push them together.
---