You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sergey Zhemzhitsky <sz...@gmail.com> on 2019/03/09 11:16:54 UTC
Mapping of CLI arguments to flink-conf.yaml
Hi there
I mostly use flink for stream-processing jobs and deploy them as
independent ones [1]. So I'd like every job to have its own
configuration and don't like to remember the values of all the command
line arguments used to start the job every time.
In spark for most of the CLI arguments there is a corresponding
configuration option that can be set on the per-job basis in
spark-defaults.conf file or programmatically via SparkConf. For
example --name CLI argument has spark.app.name configuration option,
etc.
So I'm just wondering which options from flink-conf.yaml correspond to
the following command line arguments:
Options for yarn-cluster mode:
-d,--detached If present, runs the job in detached
mode
-m,--jobmanager <arg> Address of the JobManager (master) to
which to connect. Use this flag to
connect to a different JobManager than
the one specified in the
configuration.
-yD <property=value> use value for given property
-yd,--yarndetached If present, runs the job in detached
mode (deprecated; use non-YARN
specific option instead)
-yh,--yarnhelp Help for the Yarn session CLI.
-yid,--yarnapplicationId <arg> Attach to running YARN session
-yj,--yarnjar <arg> Path to Flink jar file
-yjm,--yarnjobManagerMemory <arg> Memory for JobManager Container with
optional unit (default: MB)
-yn,--yarncontainer <arg> Number of YARN container to allocate
(=Number of Task Managers)
-ynl,--yarnnodeLabel <arg> Specify YARN node label for the YARN
application
-ynm,--yarnname <arg> Set a custom name for the application
on YARN
-yq,--yarnquery Display available YARN resources
(memory, cores)
-yqu,--yarnqueue <arg> Specify YARN queue.
-ys,--yarnslots <arg> Number of slots per TaskManager
-yst,--yarnstreaming Start Flink in streaming mode
-yt,--yarnship <arg> Ship files in the specified directory
(t for transfer)
-ytm,--yarntaskManagerMemory <arg> Memory per TaskManager Container with
optional unit (default: MB)
-yz,--yarnzookeeperNamespace <arg> Namespace to create the Zookeeper
sub-paths for high availability mode
-z,--zookeeperNamespace <arg> Namespace to create the Zookeeper
sub-paths for high availability mode
[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/yarn_setup.html#run-a-single-flink-job-on-yarn