[flink] branch master updated: [FLINK-12020] [docs] Add documentation for mesos-appmaster-job.sh

[tr...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

trohrmann pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git


The following commit(s) were added to refs/heads/master by this push:
     new b36bca8  [FLINK-12020] [docs] Add documentation for mesos-appmaster-job.sh
b36bca8 is described below

commit b36bca8aa1ceb018ecda47387ab4447941485dae
Author: jackyyin <ja...@eoitek.com>
AuthorDate: Sun Mar 31 15:31:13 2019 +0800

    [FLINK-12020] [docs] Add documentation for mesos-appmaster-job.sh
    
    1. Rename the section title from `standalone` to `Flink session cluster on Mesos`.
    2. Add one new section `Flink job cluster on Mesos` which describes the usage of mesos-appmaster-job.sh.
    
    This closes #8084.
---
 docs/ops/deployment/mesos.md | 35 ++++++++++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/docs/ops/deployment/mesos.md b/docs/ops/deployment/mesos.md
index b36bdec..8395bd0 100644
--- a/docs/ops/deployment/mesos.md
+++ b/docs/ops/deployment/mesos.md
@@ -154,7 +154,9 @@ If set to 'docker', specify the image name:
     mesos.resourcemanager.tasks.container.image.name: image_name
 
 
-### Standalone
+### Flink session cluster on Mesos
+
+A Flink session cluster is executed as a long-running Mesos Deployment. Note that you can run multiple Flink jobs on a session cluster. Each job needs to be submitted to the cluster after the cluster has been deployed.
 
 In the `/bin` directory of the Flink distribution, you find two startup scripts
 which manage the Flink processes in a Mesos cluster:
@@ -173,6 +175,37 @@ In order to run the `mesos-appmaster.sh` script you have to define `mesos.master
 When executing `mesos-appmaster.sh`, it will create a job manager on the machine where you executed the script.
 In contrast to that, the task managers will be run as Mesos tasks in the Mesos cluster.
 
+### Flink job cluster on Mesos
+
+A Flink job cluster is a dedicated cluster which runs a single job. 
+There is no extra job submission needed.
+
+In the `/bin` directory of the Flink distribution, you find one startup script 
+which manage the Flink processes in a Mesos cluster:
+
+1. `mesos-appmaster-job.sh`
+   This starts the Mesos application master which will register the Mesos scheduler, retrieve the job graph and then launch the task managers accordingly. 
+
+In order to run the `mesos-appmaster-job.sh` script you have to define `mesos.master` and `internal.jobgraph-path` in the `flink-conf.yaml` 
+or pass it via `-Dmesos.master=... -Dinterval.jobgraph-path=...` to the Java process.
+
+The job graph file may be generated like this way:
+
+{% highlight java %}
+final JobGraph jobGraph = env.getStreamGraph().getJobGraph();
+jobGraph.setAllowQueuedScheduling(true);
+final String jobGraphFilename = "job.graph";
+File jobGraphFile = new File(jobGraphFilename);
+try (FileOutputStream output = new FileOutputStream(jobGraphFile);
+	ObjectOutputStream obOutput = new ObjectOutputStream(output)){
+	obOutput.writeObject(jobGraph);
+}
+{% endhighlight %}
+
+Note: 
+1. Before serializing the job graph, please make sure to enable queued scheduling because slots need to be allocated lazily
+2. Make sure that all Mesos processes have the user code jar on the classpath (e.g. putting them in the lib directory)
+
 #### General configuration
 
 It is possible to completely parameterize a Mesos application through Java properties passed to the Mesos application master.