[jira] [Comment Edited] (FLINK-15179) Kubernetes should not have a CustomCommandLine.

["Kostas Kloudas (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16993356#comment-16993356 ] 

Kostas Kloudas edited comment on FLINK-15179 at 12/11/19 9:47 AM:
------------------------------------------------------------------

Hi [~fly_in_gis]! Thanks for jumping in the discussion.

For the {{FlinkYarnSessionCli}} I would suggest to deprecate it, as a first step, and then see if/when to remove it. Removing it may be a bit tricky because it has been there forever and many people already use it.

For the {{FlinkKubernetesCustomCli}} I would suggest to remove it and let only the full names as options, just the way you described it above. The reason is that we introduce many layers and names for the same variables, and it is difficult to keep track and even maintain the documentation up-to-date. As an example, if you check the {{Yarn}} docs, some of the CLI options are not actually supported any more. In addition, with the addition of https://issues.apache.org/jira/browse/FLINK-15121, the user will be able to specify everything programmatically in the environment, rather than the command line. So the long command lines will hopefully be rare.  In any case, if we see that this creates problems for the users, we can always re-add a short CLI options in the future, but if we add them now and people start using them, then we cannot remove them easily.

For the {{AbstractCustomCommandLine}} I would say that we can, in the future, deprecate it, and even remove it if the special Yarn CLI is also removed.

That said, could you help with testing the changes here [https://github.com/apache/flink/pull/10527]? The main change that I would appreciate to be tested more (as there is no test in the original codebase) is the parsing of the {{KubernetesConfigOptionsInternal.ENTRY_POINT_CLASS_ARGS}}. Given that this is internal and we do not want to expose it to the users, I added two public config options that we then combine in the {{KubernetesClusterDescriptor}}. 


was (Author: kkl0u):
Hi [~fly_in_gis]! Thanks for jumping in the discussion.

For the {{FlinkYarnSessionCli}} I would suggest to deprecate it, as a first step, and then see if/when to remove it. Removing it may be a bit tricky because it has been there forever and many people already use it.

For the {{FlinkKubernetesCustomCli}} I would suggest to remove it and let only the full names as options, just the way you described it above. The reason is that we introduce many layers and names for the same variables, and it is difficult to keep track and even maintain the documentation up-to-date. As an example, if you check the {{Yarn}} docs, some of the CLI options are not actually supported any more. In addition, with the addition of https://issues.apache.org/jira/browse/FLINK-15121, the user will be able to specify everything programmatically in the environment, rather than the command line. So the long command lines will hopefully be rare.  In any case, if we see that this creates problems for the users, we can always re-add a short CLI options in the future, but if we add them now and people start using them, then we cannot remove them easily.

For the {{AbstractCustomCommandLine}} I would say that we can, in the future, deprecate it, and even remove it if the special Yarn CLI is also removed.

That said, could you help with testing the changes here [https://github.com/kl0u/flink/commits/kube-pr]? The main change that I would appreciate to be tested more (as there is no test in the original codebase) is the parsing of the {{KubernetesConfigOptionsInternal.ENTRY_POINT_CLASS_ARGS}}. Given that this is internal and we do not want to expose it to the users, I added two public config options that we then combine in the {{KubernetesClusterDescriptor}}. 

> Kubernetes should not have a CustomCommandLine.
> -----------------------------------------------
>
>                 Key: FLINK-15179
>                 URL: https://issues.apache.org/jira/browse/FLINK-15179
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Deployment / Kubernetes
>    Affects Versions: 1.10.0
>            Reporter: Kostas Kloudas
>            Assignee: Kostas Kloudas
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> As part of FLIP-73, all command line options are mapped to config options. Given this 1-to-1 mapping, the Kubernetes command line could simply forward the command line arguments to ConfigOptions directly, instead of introducing new command line options. In this case, the user is expected to simply write:
>  
> {\{bin/run -e (or --executor) kubernetes-session-cluster -D kubernetes.container.image=MY_IMAGE ...}} 
> and the CLI will parse the -e to figure out the correct {{ClusterClientFactory}} and {{ExecutorFactory}} and then forward to that the config options specified with {{-D}}. 
> For this, we need to introduce a {{GenericCustomCommandLine}} that simply forward the specified parameters to the executors.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)