You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by tillrohrmann <gi...@git.apache.org> on 2018/01/02 09:16:28 UTC
[GitHub] flink pull request #5227: [FLINK-8341] [flip6] Remove not needed options fro...
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/5227
[FLINK-8341] [flip6] Remove not needed options from CommandLineOptions
## What is the purpose of the change
Remove not needed options from `CommandLineOptions`
This PR is based on #5226.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink removeNotNeededProgramOptions
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5227.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5227
----
commit c48324702b0ebff3fd7a16e480ead90fc4c9a30b
Author: Till Rohrmann <tr...@...>
Date: 2017-12-07T12:57:24Z
[FLINK-8328] [flip6] Move Yarn ApplicationStatus polling out of YarnClusterClient
Introduce YarnApplicationStatusMonitor which does the Yarn ApplicationStatus polling in
the FlinkYarnSessionCli. This decouples the YarnClusterClient from the actual communication
with Yarn and, thus, gives a better separation of concerns.
commit 783bd1daf36b34a74cbe718a52a1043ba38d5a44
Author: Till Rohrmann <tr...@...>
Date: 2017-12-20T15:43:21Z
[FLINK-8329] [flip6] Move YarnClient to AbstractYarnClusterDescriptor
Moves the YarnClient from the YarnClusterClient to the AbstractYarnClusterDescriptor.
This makes the latter responsible for the lifecycle management of the client and gives
a better separation of concerns.
commit 192adb786a48d19b71e797355500652d51de6296
Author: Till Rohrmann <tr...@...>
Date: 2017-12-18T17:59:30Z
[FLINK-8332] [flip6] Move savepoint dispose into ClusterClient
Move the savepoint disposal logic from the CliFrontend into the ClusterClient. This gives
a better separation of concerns and allows the CliFrontend to be used with different
ClusterClient implementations.
commit a73b6e2850d4b2445a835914ba570d1057e59dfb
Author: Till Rohrmann <tr...@...>
Date: 2018-01-02T06:42:18Z
[FLINK-8333] [flip6] Separate deployment options from command options
This commit separates the parsing of command options and deployment options into two
steps. This makes it easier to make the CustomCommandLines non-static.
Moreover, this commit moves the CliFrontend into the cli sub package.
commit e32fa9d4eb45bfac2d13ab112f127be29d273ebd
Author: Till Rohrmann <tr...@...>
Date: 2018-01-02T06:59:34Z
[FLINK-8338] [flip6] Make CustomCommandLines non static in CliFrontend
This commit changes how CustomCommandLines are registered at the CliFrontend.
Henceforth, the CliFrontend is initialized with the set of CustomCommandLines
instead of registering them statically. This improves maintainability and
testability.
commit 7972fff6e574f53b057eef6e5326d97f1129af66
Author: Till Rohrmann <tr...@...>
Date: 2018-01-02T08:22:12Z
[FLINK-8339] [flip6] Let CustomCommandLine return ClusterDescriptor
Instead of directly retrieving or deploying a Flink cluster, the
CustomCommandLine now only returns a ClusterDescriptor which can be used
for these operations. This disentangles the ClusterDescriptor and the
CustomCommandLine a bit better supporting a proper lifecycle management
of the former.
commit 061d7de4b23b20fb090ec345b7e089da766a8c6a
Author: Till Rohrmann <tr...@...>
Date: 2018-01-02T08:55:13Z
[FLINK-8340] [flip6] Remove passing of Configuration to CustomCommandLine
Since the Configuration does not change over the lifetime of a CustomCommandLine,
we can safely pass it as a constructor argument instead of method argument.
commit ec516c14dcb296146c61585c496016ab7982df8d
Author: Till Rohrmann <tr...@...>
Date: 2017-12-20T15:32:18Z
[FLINK-8341] [flip6] Remove not needed options from CommandLineOptions
----
---
[GitHub] flink pull request #5227: [FLINK-8341] [flip6] Remove not needed options fro...
Posted by GJL <gi...@git.apache.org>.
Github user GJL commented on a diff in the pull request:
https://github.com/apache/flink/pull/5227#discussion_r160113577
--- Diff: flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkShell.scala ---
@@ -281,16 +287,22 @@ object FlinkShell {
"-m", "yarn-cluster"
)
- val options = CliFrontendParser.parseRunCommand(args.toArray)
+ val commandLine = CliFrontendParser.parse(
+ CliFrontendParser.getRunCommandOptions,
+ args.toArray,
+ true)
+
+ val options = new RunOptions(commandLine)
+
val frontend = new CliFrontend(
configuration,
CliFrontend.loadCustomCommandLines(configuration, configurationDirectory))
val config = frontend.getConfiguration
--- End diff --
Unused variable `config`.
---
[GitHub] flink pull request #5227: [FLINK-8341] [flip6] Remove not needed options fro...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5227
---
[GitHub] flink issue #5227: [FLINK-8341] [flip6] Remove not needed options from Comma...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/5227
Thanks for the review @GJL. Addressing comments and merging once Travis gives green light.
---
[GitHub] flink pull request #5227: [FLINK-8341] [flip6] Remove not needed options fro...
Posted by GJL <gi...@git.apache.org>.
Github user GJL commented on a diff in the pull request:
https://github.com/apache/flink/pull/5227#discussion_r160112134
--- Diff: flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkShell.scala ---
@@ -253,16 +253,22 @@ object FlinkShell {
yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString))
yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString))
- val options = CliFrontendParser.parseRunCommand(args.toArray)
+ val commandLine = CliFrontendParser.parse(
+ CliFrontendParser.getRunCommandOptions,
+ args.toArray,
+ true)
+
+ val options = new RunOptions(commandLine)
--- End diff --
Unused variable.
---
[GitHub] flink pull request #5227: [FLINK-8341] [flip6] Remove not needed options fro...
Posted by GJL <gi...@git.apache.org>.
Github user GJL commented on a diff in the pull request:
https://github.com/apache/flink/pull/5227#discussion_r160113403
--- Diff: flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkShell.scala ---
@@ -253,16 +253,22 @@ object FlinkShell {
yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString))
yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString))
- val options = CliFrontendParser.parseRunCommand(args.toArray)
+ val commandLine = CliFrontendParser.parse(
+ CliFrontendParser.getRunCommandOptions,
+ args.toArray,
+ true)
+
+ val options = new RunOptions(commandLine)
+
val frontend = new CliFrontend(
configuration,
CliFrontend.loadCustomCommandLines(configuration, configurationDirectory))
val config = frontend.getConfiguration
--- End diff --
Unused variable `config`.
---
[GitHub] flink pull request #5227: [FLINK-8341] [flip6] Remove not needed options fro...
Posted by GJL <gi...@git.apache.org>.
Github user GJL commented on a diff in the pull request:
https://github.com/apache/flink/pull/5227#discussion_r160113517
--- Diff: flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkShell.scala ---
@@ -281,16 +287,22 @@ object FlinkShell {
"-m", "yarn-cluster"
)
- val options = CliFrontendParser.parseRunCommand(args.toArray)
+ val commandLine = CliFrontendParser.parse(
+ CliFrontendParser.getRunCommandOptions,
+ args.toArray,
+ true)
+
+ val options = new RunOptions(commandLine)
--- End diff --
Unused variable `options`.
---