You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/01/08 10:54:10 UTC

[GitHub] [flink] yanghua commented on a change in pull request #10777: [FLINK-10939][doc] Add documents for running Flink cluster natively on Kubernetes

yanghua commented on a change in pull request #10777: [FLINK-10939][doc] Add documents for running Flink cluster natively on Kubernetes
URL: https://github.com/apache/flink/pull/10777#discussion_r364170639
 
 

 ##########
 File path: docs/ops/deployment/native_kubernetes.md
 ##########
 @@ -0,0 +1,199 @@
+---
+title:  "Native Kubernetes Setup"
+nav-title: Native Kubernetes
+nav-parent_id: deployment
+is_beta: true
+nav-pos: 7
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This page describes how to deploy a Flink session cluster natively on [Kubernetes](https://kubernetes.io).
+
+* This will be replaced by the TOC
+{:toc}
+
+<div class="alert alert-warning">
+Flink's native Kubernetes integration is still experimental. There may be changes in the configuration and CLI flags in latter versions. Job clusters are not yet supported.
+</div>
+
+## Requirements
+
+- Kubernetes 1.9 or above.
+- KubeConfig, which has access to list, create, delete pods and services, configurable via `~/.kube/config`. You can verify permissions by running `kubectl auth can-i <list|create|edit|delete> pods`.
+- Kubernetes DNS enabled.
+- A service Account with [RBAC](#rbac) permissions to create, delete pods.
+
+## Flink Kubernetes Session
+
+### Start Flink Session
+
+Follow these instructions to start a Flink Session within your Kubernetes cluster.
+
+A session will start all required Flink services (JobManager and TaskManagers) so that you can submit programs to the cluster.
+Note that you can run multiple programs per session.
+
+{% highlight bash %}
+$ ./bin/kubernetes-session.sh
+{% endhighlight %}
+
+All the Kubernetes configuration options can be found in our [configuration guide]({{ site.baseurl }}/ops/config.html#kubernetes).
+
+**Example**: Issue the following command to start a session cluster with 4 GB of memory and 2 CPUs with 4 slots per TaskManager:
+
+{% highlight bash %}
+./bin/kubernetes-session.sh \
+  -Dkubernetes.cluster-id=<ClusterId> \
+  -Dtaskmanager.memory.process.size=4096m \
+  -Dkubernetes.taskmanager.cpu=2 \
+  -Dtaskmanager.numberOfTaskSlots=4
+{% endhighlight %}
+
+The system will use the configuration in `conf/flink-conf.yaml`.
+Please follow our [configuration guide]({{ site.baseurl }}/ops/config.html) if you want to change something.
+
+If you do not specify a particular name for your session by `kubernetes.cluster-id`, the Flink client will generate a UUID name. 
+
+### Submitting jobs to an existing Session
+
+Use the following command to submit a Flink Job to the Kubernetes cluster.
+
+{% highlight bash %}
+$ ./bin/flink run -d -e kubernetes-session -Dkubernetes.cluster-id=<ClusterId> examples/streaming/WindowJoin.jar
+{% endhighlight %}
+
+### Accessing Job Manager UI
+
+There are several ways to expose a Service onto an external (outside of your cluster) IP address.
+This can be configured using `kubernetes.service.exposed.type`.
+
+- `ClusterIP`: Exposes the service on a cluster-internal IP.
+The Service is only reachable within the cluster. If you want to access the Job Manager ui or submit job to the existing session, you need to start a local proxy.
 
 Review comment:
   ui -> UI and `submit job` -> `submit a job`?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services