You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by NicoK <gi...@git.apache.org> on 2017/06/30 15:10:40 UTC
[GitHub] flink pull request #4237: [FLINK-7056][blob] add API to allow job-related BL...
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/4237
[FLINK-7056][blob] add API to allow job-related BLOBs to be stored
To ease cleanup, we will make job-related BLOBs be reflected in the blob storage so that they may be removed along with the job. This adds the `jobId` to many methods similar to the previous code from the `NAME_ADDRESSABLE` mode.
This PR is based upon #4236 in a series to implement FLINK-6916.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-6916-7056
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4237.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4237
----
commit d54a316cfffd8243980df561fd4fcbd99934a40b
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2016-12-20T15:49:57Z
[FLINK-6008][docs] minor improvements in the BlobService docs
commit b215515fa14d3f6af218e86b67bc2c27ae9d4f4f
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2016-12-20T17:27:13Z
[FLINK-6008] refactor BlobCache#getURL() for cleaner code
commit bbcde52b3105fcf379c852b568f3893cc6052ce6
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2016-12-21T15:23:29Z
[FLINK-6008] do not fail the BlobServer if delete fails
also extend the delete tests and remove one code duplication
commit dda1a12e40027724efb0e50005e5b57058a220f0
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-01-06T17:42:58Z
[FLINK-6008][docs] update some config options to the new, non-deprecated ones
commit e12c2348b237207a50649d515a0fbbd19f92e6a0
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-03-09T17:14:02Z
[FLINK-6008] use Preconditions.checkArgument in BlobClient
commit 24060e01332c6df9fd01f1dc5f321c3fda9301c1
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-03-17T15:21:40Z
[FLINK-6008] fix concurrent job directory creation
also add according unit tests
commit 2e0d16ab8bf8a48a2d028602a3a7693fc4b76039
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-14T16:01:47Z
[FLINK-6008] do not guard a delete() call with a check for existence
commit 7ba911d7ecb4861261dff8509996be0bd64d6d27
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-04-18T14:37:37Z
[FLINK-6008] some comments about BlobLibraryCacheManager cleanup
commit d3f50d595f85356ae6ed0a85e1f8b8e8ac630bde
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-04-19T13:39:03Z
[hotfix] minor typos
commit 79b6ce35a9e246b35415a388295f9ee2fc19a82e
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-04-19T14:10:16Z
[FLINK-6008] further cleanup tests for BlobLibraryCacheManager
commit 23fb6ecd6c43c86d762503339c67953290236dca
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-30T14:03:16Z
[FLINK-6008] address PR comments
commit 794764ceeed6b9bbbac08662f5754b218ff86c9c
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-16T08:51:04Z
[FLINK-7052][blob] remove (unused) NAME_ADDRESSABLE mode
commit 774bafa85f242110a2ce7907c1150f8c62d73b3f
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T15:05:57Z
[FLINK-7052][blob] remove further unused code due to the NAME_ADDRESSABLE removal
commit 4da3b3f6269e43bf1c66621099528824cad9373f
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-22T15:31:17Z
[FLINK-7053][blob] remove code duplication in BlobClientSslTest
This lets BlobClientSslTest extend BlobClientTest as most of its implementation
came from there and was simply copied.
commit aa9cdc820f9ca1a38a19708bf45a2099e42eaf48
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-23T09:40:34Z
[FLINK-7053][blob] verify some of the buffers returned by GET
commit c9b693a46053b55b3939ff471184796f12d36a72
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-23T10:04:10Z
[FLINK-7053][blob] use TemporaryFolder for local BLOB dir in unit tests
This replaces the use of some temporary directory where it is not guaranteed
that it will be deleted after the test.
commit 11db399d5103d9ffe9083c9b6029a7e81afa9abe
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T12:45:31Z
[FLINK-7054][blob] remove LibraryCacheManager#getFile()
This was only used in tests where it is avoidable but if used anywhere else, it
may have caused cleanup issues.
commit 4ae04b68453d4b099f752d6c6fd3c09335ede33a
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T14:14:15Z
[FLINK-7055][blob] refactor getURL() to the more generic getFile()
The fact that we always returned URL objects is a relic of the BlobServer's only
use for URLClassLoader. Since we'd like to extend its use, returning File
objects instead is more generic.
commit 8397d6aa5dc0aac07626d0af9ee3d8623dd7b60c
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T16:04:43Z
[FLINK-7056][blob] add API to allow job-related BLOBs to be stored
commit 0a4c4e9bc483e4f1f885ef1e3b8feba40c057204
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-23T17:17:07Z
[FLINK-7056][blob] refactor the new API for job-related BLOBs
For a cleaner API, instead of having a nullable jobId parameter, use two methods:
one for job-related BLOBs, another for job-unrelated ones.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4237: [FLINK-7056][blob] add API to allow job-related BL...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4237
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---