[jira] [Commented] (FLINK-5494) Improve Mesos documentation

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849572#comment-15849572 ] 

ASF GitHub Bot commented on FLINK-5494:
---------------------------------------

Github user tzulitai commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3236#discussion_r99061517
  
    --- Diff: docs/setup/mesos.md ---
    @@ -145,60 +174,75 @@ If set to 'docker', specify the image name:
     In the `/bin` directory of the Flink distribution, you find two startup scripts
     which manage the Flink processes in a Mesos cluster:
     
    -1. mesos-appmaster.sh
    -   This starts the Mesos application master which will register the Mesos
    -   scheduler. It is also responsible for starting up the worker nodes.
    +1. `mesos-appmaster.sh`
    +   This starts the Mesos application master which will register the Mesos scheduler.
    +   It is also responsible for starting up the worker nodes.
        
    -2. mesos-taskmanager.sh
    -   The entry point for the Mesos worker processes. You don't need to explicitly
    -   execute this script. It is automatically launched by the Mesos worker node to
    -   bring up a new TaskManager.
    +2. `mesos-taskmanager.sh`
    +   The entry point for the Mesos worker processes.
    +   You don't need to explicitly execute this script.
    +   It is automatically launched by the Mesos worker node to bring up a new TaskManager.
    +
    +In order to run the `mesos-appmaster.sh` script you have to define `mesos.master` in the `flink-conf.yaml` or pass it via `-Dmesos.master=...` to the Java process.
    +Additionally, you should define the number of task managers which are started by Mesos via `mesos.initial-tasks`.
    +This value can also be defined in the `flink-conf.yaml` or passed as a Java property.
    +
    +When executing `mesos-appmaster.sh`, it will create a job manager on the machine where you executed the script.
    +In contrast to that, the task managers will be run as Mesos tasks in the Mesos cluster.
    +
    +#### General configuration
    +
    +It is possible to completely parameterize a Mesos application through Java properties passed to the Mesos application master.
    +This also allows to specify general Flink configuration parameters.
    +For example:
    +
    +    bin/mesos-appmaster.sh \
    +        -Dmesos.master=master.foobar.org:5050
    +        -Djobmanager.heap.mb=1024 \
    +        -Djobmanager.rpc.port=6123 \
    +        -Djobmanager.web.port=8081 \
    +        -Dmesos.initial-tasks=10 \
    +        -Dmesos.resourcemanager.tasks.mem=4096 \
    +        -Dtaskmanager.heap.mb=3500 \
    +        -Dtaskmanager.numberOfTaskSlots=2 \
    +        -Dparallelism.default=10
     
     
     ### High Availability
     
    -You will need to run a service like Marathon or Apache Aurora which takes care
    -of restarting the Flink master process in case of node or process failures. In
    -addition, Zookeeper needs to be configured like described in the
    -[High Availability section of the Flink docs]({{ site.baseurl }}/setup/jobmanager_high_availability.html)
    +You will need to run a service like Marathon or Apache Aurora which takes care of restarting the Flink master process in case of node or process failures.
    +In addition, Zookeeper needs to be configured like described in the [High Availability section of the Flink docs]({{ site.baseurl }}/setup/jobmanager_high_availability.html)
     
    -For the reconciliation of tasks to work correctly, please also set
    -`recovery.zookeeper.path.mesos-workers` to a valid Zookeeper path.
    +For the reconciliation of tasks to work correctly, please also set `recovery.zookeeper.path.mesos-workers` to a valid Zookeeper path.
     
     #### Marathon
     
    -Marathon needs to be set up to launch the `bin/mesos-appmaster.sh` script. In
    -particular, it should also adjust any configuration parameters for the Flink
    -cluster. 
    +Marathon needs to be set up to launch the `bin/mesos-appmaster.sh` script.
    +In particular, it should also adjust any configuration parameters for the Flink cluster.
     
     Here is an example configuration for Marathon:
     
         {
    -    "id": "basic-0", 
    -    "cmd": "$FLINK_HOME/bin/mesos-appmaster.sh -DconfigEntry=configValue -DanotherEntry=anotherValue ...",
    +    "id": "flink",
    +    "cmd": "$FLINK_HOME/bin/mesos-appmaster.sh -Djobmanager.heap.mb=1024 -Djobmanager.rpc.port=6123 -Djobmanager.web.port=8081 -Dmesos.initial-tasks=1 -Dmesos.resourcemanager.tasks.mem=1024 -Dtaskmanager.heap.mb=1024 -Dtaskmanager.numberOfTaskSlots=2 -Dparallelism.default=2 -Dmesos.resourcemanager.tasks.cpus=1",
         "cpus": 1.0,
    -    "mem": 2048,
    +    "mem": 1024
         }
     
    +When running Flink with Marathon, the whole Flink cluster including the job manager will be run as Mesos tasks in the Mesos cluster.
    +
     ### Configuration parameters
     
     #### Mesos configuration entries
    --- End diff --
    
    Should we add the Mesos config to `setup/config.md` as well?


> Improve Mesos documentation
> ---------------------------
>
>                 Key: FLINK-5494
>                 URL: https://issues.apache.org/jira/browse/FLINK-5494
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Mesos
>    Affects Versions: 1.2.0
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
>             Fix For: 1.2.0
>
>
> Flink's Mesos documentation could benefit from more details how to set things up and which parameters to use.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)