You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by NicoK <gi...@git.apache.org> on 2017/06/23 22:06:16 UTC
[GitHub] flink pull request #4176: [FLINK-6916][blob] add API to allow job-related BL...
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/4176
[FLINK-6916][blob] add API to allow job-related BLOBs to be stored
This PR is the fourth in a series for FLIP-19, based upon #4174. It allows (CONTENT_ADDRESSABLE) BLOBs to not only be stored job-independently but also based on a jobId which will become the future default for JARs, `TaskDeploymentDescriptor` data, and `DistributedCache` data.
Please note that we will rework/reorganise the whole BLOB store APIs completely in a future request. This PR focusses on the added functionality.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-6916-3-jobid
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4176.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4176
----
commit ce719ee39fbbca7b7828c17d9792fc87d37450c7
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-01-06T17:42:58Z
[FLINK-6008][docs] update some config options to the new, non-deprecated ones
commit 9efa8808e46adc1253f52a6a8cec6d3b4d29fee3
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2016-12-20T15:49:57Z
[FLINK-6008][docs] minor improvements in the BlobService docs
commit ca3d533b0affa645ec93d40de378dadc829bbfe5
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2016-12-20T17:27:13Z
[FLINK-6008] refactor BlobCache#getURL() for cleaner code
commit 0eededeb36dd833835753def7f4bb27c9d5fb67e
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-03-09T17:14:02Z
[FLINK-6008] use Preconditions.checkArgument in BlobClient
commit 6249041a9db2b39ddf54e79a1aed5e7706e739c7
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2016-12-21T15:23:29Z
[FLINK-6008] do not fail the BlobServer if delete fails
also extend the delete tests and remove one code duplication
commit e681239a538547f752d65358db1ebd2ba312b33c
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-03-17T15:21:40Z
[FLINK-6008] fix concurrent job directory creation
also add according unit tests
commit 20beae2dbc91859e2ec724b35b20536dcd11fe90
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-04-18T14:37:37Z
[FLINK-6008] some comments about BlobLibraryCacheManager cleanup
commit 8a33517fe6eb2fa932ab17cb0d82a3fa8d7b8d0b
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-04-19T13:39:03Z
[hotfix] minor typos
commit 23889866ac21494fc4af90905ab1518cbe897118
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-04-19T14:10:16Z
[FLINK-6008] further cleanup tests for BlobLibraryCacheManager
commit 01b1a245528c264a6061ed3a48b24c5a207369f6
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-14T16:01:47Z
[FLINK-6008] do not guard a delete() call with a check for existence
commit cb249759b79d88eda37a8bb149040be3052059ac
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-16T08:51:04Z
[FLINK-6916][blob] remove (unused) NAME_ADDRESSABLE mode
commit 7d3333c9cdb5bcf19a5e6b3190ce564201f1cc24
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T15:05:57Z
[FLINK-6916][blob] remove further unused code due to the NAME_ADDRESSABLE removal
commit 00242371fed84a658ce88765204c450cc7819cf3
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-22T15:31:17Z
[FLINK-6916] remove code duplication in BlobClientSslTest
This lets BlobClientSslTest extend BlobClientTest as most of its implementation
came from there and was simply copied.
commit 2a251e5cef0b757333d0dc1ff37f78fa0ea6eba7
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T12:45:31Z
[FLINK-6916] remove LibraryCacheManager#getFile()
This was only used in tests where it is avoidable but if used anywhere else, it
may have caused cleanup issues.
commit 12d76e96247fc34a498260d323c67cbd07f3f905
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T14:14:15Z
[FLINK-6916][blob] refactor getURL() to the more generic getFile()
The fact that we always returned URL objects is a relic of the BlobServer's only
use for URLClassLoader. Since we'd like to extend its use, returning File
objects instead is more generic.
commit 3126c3c5e0c146e16d415fe75ccbfaf1770a3bea
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-23T09:40:34Z
[FLINK-6916][blob] verify some of the buffers returned by GET
commit 4992ad7bb093444356d4258f2abe0731e1c18825
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-23T10:04:10Z
[FLINK-6916][blob] use TemporaryFolder for local BLOB dir in unit tests
This replaces the use of some temporary directory where it is not guaranteed
that it will be deleted after the test.
commit d87d742b9a19efc1a26d2dda0c724d286ba9904d
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-21T16:04:43Z
[FLINK-6916][blob] add API to allow job-related BLOBs to be stored
commit 053f4ba218e10038ddb4fa8790bb171144f70a72
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-06-23T17:17:07Z
[FLINK-6916][blob] refactor the new API for job-related BLOBs
For a cleaner API, instead of having a nullable jobId parameter, use two methods:
one for job-related BLOBs, another for job-unrelated ones.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4176: [FLINK-6916][blob] add API to allow job-related BL...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK closed the pull request at:
https://github.com/apache/flink/pull/4176
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4176: [FLINK-6916][blob] add API to allow job-related BLOBs to ...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4176
superseeded by #4237
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---