You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by tillrohrmann <gi...@git.apache.org> on 2017/07/07 14:02:49 UTC
[GitHub] flink pull request #4284: [FLINK-7080] [yarn] Add Yarn per job mode deployme...
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/4284
[FLINK-7080] [yarn] Add Yarn per job mode deployment
This PR is based on #4280, #4270, #4271, #4259, #4260, #4261, #4272 and #4281.
This PR adds the functionality to deploy a Yarn per-job cluster via the `YarnClusterDescriptorV2.deployJob(ClusterSpecification, JobGraph)`. The `JobGraph` has to contain the user code jar which is required to run the job.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink addPerJobYarnDeployment
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4284.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4284
----
commit b415fdba6334aa8eeed9fecb3834dbce5ceecc1e
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-07T11:40:29Z
[FLINK-7125] [yarn] Remove Configuration loading from AbstractYarnClusterDescriptor
Instead the AbstractYarnClusterDescriptor is passed in a Configuration instance which
is sent to the started application master.
Pass in configuration directory manually
Remove configurationDirectory resolution from AbstractYarnClusterDescriptor
commit 8de588bf3b2bc984ecc25603e474df443f5cc265
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-06T07:33:18Z
[FLINK-7110] [client] Add per-job cluster deployment interface
commit c68942eb7e154e78bef16930fa8f421f0123483e
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-06T12:00:21Z
[FLINK-7113] Make ClusterDescriptor independent of cluster size
The deploySession method now is given a ClusterSpecification which specifies the
size of the cluster which it is supposed to deploy.
commit 7003cae3c7530b6560dd8ebd365b1b1313a109e0
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-04T21:15:52Z
[FLINK-7103] [dispatcher] Add skeletal structure of Dispatcher component
The Dispatcher is responsible for receiving job submissions, persisting the JobGraphs,
spawning JobManager to execute the jobs and recovering the jobs in case of a master
failure. This commit adds the basic skeleton including the RPC call for job submission.
Add cleanup logic for finished jobs
Pass BlobService to JobManagerRunner
commit 24fa27535d7064cd2018c67969f9d2c2ef0238f3
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-05T11:38:35Z
[FLINK-7105] Make ActorSystems non daemonic
In order to not having to explicitly wait on the termination of an ActorSystem
in the main thread, we now create the ActorSystems in non-daemonic mode. That
way the process won't terminate if there is still an active actor.
commit 60294e77b5d9cb91b9e31c0a80d3514d32c80c88
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-05T15:16:52Z
[FLINK-7108] [yarn] Add YARN entry points based on the generic entry point
Add the YarnSesssionClusterEntrypoint and the YarnJobClusterEntrypoint which extend
SessionClusterEntrypoint and JobClusterEntrypoint, respectively.
Add new Yarn session and per-job cluster entry points
Remove old Flip-6 Yarn per job entry point
commit dee79e55de3575cacdc9ce0d95a4c43974c2d0f6
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-06T15:16:18Z
[FLINK-7082] Add generic entry point for session and per-job clusters
commit a940663e7e2894dc1ddd365ca6ae539b52a85c65
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-06T15:16:54Z
[FLINK-7086] Add Flip-6 standalone session cluster entry point
commit fce951241e3a4723a6c32b6c08b1aea7513ec397
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-06T16:26:41Z
[FLINK-7080] [yarn] Add Yarn per job mode deployment
Upload user code jar from JobGraph
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4284: [FLINK-7080] [yarn] Add Yarn per job mode deployment
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4284
Rebasing this PR onto the latest #4281 commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4284: [FLINK-7080] [yarn] Add Yarn per job mode deployment
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4284
Travis passed. Merging this PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4284: [FLINK-7080] [yarn] Add Yarn per job mode deployme...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4284
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---