You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by NicoK <gi...@git.apache.org> on 2017/08/17 10:12:05 UTC
[GitHub] flink pull request #4558: [FLINK-7057][tests][hotfix] make sure the client a...
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/4558
[FLINK-7057][tests][hotfix] make sure the client and a created InputStream are closed
## What is the purpose of the change
This fixes some stalling tests in the following cases: if the client is not closed and the inputstream of a GET operation was not fully read and the server has not yet sent all data packets, it may still hold the read lock and block any writing operations (also see FLINK-7467). As a result, a user reported `BlobServerDeleteTest#testDeleteSingleByBlobKey()` to hang.
## Brief change log
- (fully read and) close all `InputStream` instances returned by `BlobClient#get()`
- add missing `close` calls to `BlobClient` uses in the tests
## Verifying this change
This change is affects and fixes existing tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-6916-7057-hotfix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4558.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4558
----
commit f80025773b071a67fdcbe612d58ee16ee7cbbc52
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-08-17T10:04:09Z
[FLINK-7057][tests][hotfix] make sure the client and a created InputStream are closed
If not and the server has not yet sent all data packets, it may still occupy the
read lock and block any writing operations (also see FLINK-7467).
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4558: [FLINK-7056][tests][hotfix] make sure the client a...
Posted by pnowojski <gi...@git.apache.org>.
Github user pnowojski commented on a diff in the pull request:
https://github.com/apache/flink/pull/4558#discussion_r133674886
--- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/blob/BlobClientTest.java ---
@@ -139,30 +139,35 @@ private static BlobKey prepareTestFile(File file) throws IOException {
* the specified buffer.
*
* @param inputStream
- * the input stream returned from the GET operation
+ * the input stream returned from the GET operation (will be closed by this method)
* @param buf
* the buffer to compare the input stream's data to
* @throws IOException
* thrown if an I/O error occurs while reading the input stream
*/
static void validateGet(final InputStream inputStream, final byte[] buf) throws IOException {
--- End diff --
`s/validateGet/validateGetAndClose/g` - to avoid surprises for the function users :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4558: [FLINK-7056][tests][hotfix] make sure the client a...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4558#discussion_r133675837
--- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/blob/BlobClientTest.java ---
@@ -139,30 +139,35 @@ private static BlobKey prepareTestFile(File file) throws IOException {
* the specified buffer.
*
* @param inputStream
- * the input stream returned from the GET operation
+ * the input stream returned from the GET operation (will be closed by this method)
* @param buf
* the buffer to compare the input stream's data to
* @throws IOException
* thrown if an I/O error occurs while reading the input stream
*/
static void validateGet(final InputStream inputStream, final byte[] buf) throws IOException {
--- End diff --
ok, why not... ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4558: [FLINK-7056][tests][hotfix] make sure the client a...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4558
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---