You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by tr...@apache.org on 2021/03/01 09:44:31 UTC
[flink] 06/06: [FLINK-15656][k8s][doc] Add documentation for pod
template
This is an automated email from the ASF dual-hosted git repository.
trohrmann pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 24fe5ab0f501647f8be199e72fcb9a2920be4fe9
Author: wangyang0918 <da...@alibaba-inc.com>
AuthorDate: Fri Feb 5 13:28:23 2021 +0800
[FLINK-15656][k8s][doc] Add documentation for pod template
This closes #14629.
---
.../resource-providers/native_kubernetes.md | 236 +++++++++++++++++++++
.../resource-providers/native_kubernetes.md | 236 +++++++++++++++++++++
2 files changed, 472 insertions(+)
diff --git a/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md b/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md
index 291b0dd..a990c4f 100644
--- a/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md
+++ b/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md
@@ -324,4 +324,240 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit
Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information.
+### Pod Template
+
+Flink allows users to define the JobManager and TaskManager pods via template files. This allows to support advanced features
+that are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly.
+Use [`kubernetes.pod-template-file`]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file)
+to specify a local file that contains the pod definition. It will be used to initialize the JobManager and TaskManager.
+The main container should be defined with name `flink-main-container`.
+Please refer to the [pod template example](#example-of-pod-template) for more information.
+
+#### Fields Overwritten by Flink
+
+Some fields of the pod template will be overwritten by Flink.
+The mechanism for resolving effective field values can be categorized as follows:
+* **Defined by Flink:** User cannot configure it.
+* **Defined by the user:** User can freely specify this value. Flink framework won't set any additional values and the effective value derives from the config option and the template.
+
+ Precedence order: First an explicit config option value is taken, then the value in pod template and at last the default value of a config option if nothing is specified.
+* **Merged with Flink:** Flink will merge values for a setting with a user defined value (see precedence order for "Defined by the user"). Flink values have precedence in case of same name fields.
+
+Refer to the following tables for the full list of pod fields that will be overwritten.
+All the fields defined in the pod template that are not listed in the tables will be unaffected.
+
+**Pod Metadata**
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 10%">Key</th>
+ <th class="text-left" style="width: 20%">Category</th>
+ <th class="text-left" style="width: 30%">Related Config Options</th>
+ <th class="text-left" style="width: 40%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>name</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>The JobManager pod name will be overwritten with the deployment which is defined by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-cluster-id">kubernetes.cluster-id</a>.
+ The TaskManager pod names will be overwritten with the pattern <code><clusterID>-<attempt>-<index></code> which is generated by Flink ResourceManager.</td>
+ </tr>
+ <tr>
+ <td>namespace</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-namespace">kubernetes.namespace</a></td>
+ <td>Both the JobManager deployment and TaskManager pods will be created in the user specified namespace.</td>
+ </tr>
+ <tr>
+ <td>ownerReferences</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>The owner reference of JobManager and TaskManager pods will always be set to the JobManager deployment.
+ Please use <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-owner-reference">kubernetes.jobmanager.owner.reference</a> to control when the deployment is deleted.</td>
+ </tr>
+ <tr>
+ <td>annotations</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-annotations">kubernetes.jobmanager.annotations</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-annotations">kubernetes.taskmanager.annotations</a></td>
+ <td>Flink will add additional annotations specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>labels</td>
+ <td>Merged with Flink</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-labels">kubernetes.jobmanager.labels</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-labels">kubernetes.taskmanager.labels</a></td>
+ <td>Flink will add some internal labels to the user defined values.</td>
+ </tr>
+ </tbody>
+</table>
+
+**Pod Spec**
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 10%">Key</th>
+ <th class="text-left" style="width: 20%">Category</th>
+ <th class="text-left" style="width: 30%">Related Config Options</th>
+ <th class="text-left" style="width: 40%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>imagePullSecrets</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-secrets">kubernetes.container.image.pull-secrets</a></td>
+ <td>Flink will add additional pull secrets specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>nodeSelector</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-node-selector">kubernetes.jobmanager.node-selector</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-node-selector">kubernetes.taskmanager.node-selector</a></td>
+ <td>Flink will add additional node selectors specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>tolerations</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-tolerations">kubernetes.jobmanager.tolerations</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-tolerations">kubernetes.taskmanager.tolerations</a></td>
+ <td>Flink will add additional tolerations specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>restartPolicy</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>"always" for JobManager pod and "never" for TaskManager pod.
+ <br>
+ The JobManager pod will always be restarted by deployment. And the TaskManager pod should not be restarted.</td>
+ </tr>
+ <tr>
+ <td>serviceAccount</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-service-account">kubernetes.service-account</a></td>
+ <td>The JobManager and TaskManager pods will be created with the user defined service account.</td>
+ </tr>
+ <tr>
+ <td>volumes</td>
+ <td>Merged with Flink</td>
+ <td></td>
+ <td>Flink will add some internal ConfigMap volumes(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td>
+ </tr>
+ </tbody>
+</table>
+
+**Main Container Spec**
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 10%">Key</th>
+ <th class="text-left" style="width: 20%">Category</th>
+ <th class="text-left" style="width: 30%">Related Config Options</th>
+ <th class="text-left" style="width: 40%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>env</td>
+ <td>Merged with Flink</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.master.env.{ENV_NAME}</a>
+ <a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.taskmanager.env.{ENV_NAME}</a></td>
+ <td>Flink will add some internal environment variables to the user defined values.</td>
+ </tr>
+ <tr>
+ <td>image</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image">kubernetes.container.image</a></td>
+ <td>The container image will be resolved with respect to the defined precedence order for user defined values.</td>
+ </tr>
+ <tr>
+ <td>imagePullPolicy</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-policy">kubernetes.container.image.pull-policy</a></td>
+ <td>The container image pull policy will be resolved with respect to the defined precedence order for user defined values.</td>
+ </tr>
+ <tr>
+ <td>name</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>The container name will be overwritten by Flink with "flink-main-container".</td>
+ </tr>
+ <tr>
+ <td>resources</td>
+ <td>Defined by the user</td>
+ <td>Memory: <br>
+ <a href="{{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size">jobmanager.memory.process.size</a>
+ <a href="{{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size">taskmanager.memory.process.size</a>
+ <br>
+ CPU: <br>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-cpu">kubernetes.jobmanager.cpu</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-cpu">kubernetes.taskmanager.cpu</a></td>
+ <td>The memory and cpu resources(including requests and limits) will be overwritten by Flink configuration options. All other resources(e.g. ephemeral-storage) will be retained.</td>
+ </tr>
+ <tr>
+ <td>containerPorts</td>
+ <td>Merged with Flink</td>
+ <td></td>
+ <td>Flink will add some internal container ports(e.g. rest, jobmanager-rpc, blob, taskmanager-rpc).</td>
+ </tr>
+ <tr>
+ <td>volumeMounts</td>
+ <td>Merged with Flink</td>
+ <td></td>
+ <td>Flink will add some internal volume mounts(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td>
+ </tr>
+ </tbody>
+</table>
+
+#### Example of Pod Template
+`pod-template.yaml`
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+ name: jobmanager-pod-template
+spec:
+ initContainers:
+ - name: artifacts-fetcher
+ image: artifacts-fetcher:latest
+ # Use wget or other tools to get user jars from remote storage
+ command: [ 'wget', 'https://path/of/StateMachineExample.jar', '-O', '/flink-artifact/myjob.jar' ]
+ volumeMounts:
+ - mountPath: /flink-artifact
+ name: flink-artifact
+ containers:
+ # Do not change the main container name
+ - name: flink-main-container
+ resources:
+ requests:
+ ephemeral-storage: 2048Mi
+ limits:
+ ephemeral-storage: 2048Mi
+ volumeMounts:
+ - mountPath: /opt/flink/volumes/hostpath
+ name: flink-volume-hostpath
+ - mountPath: /opt/flink/artifacts
+ name: flink-artifact
+ - mountPath: /opt/flink/log
+ name: flink-logs
+ # Use sidecar container to push logs to remote storage or do some other debugging things
+ - name: sidecar-log-collector
+ image: sidecar-log-collector:latest
+ command: [ 'command-to-upload', '/remote/path/of/flink-logs/' ]
+ volumeMounts:
+ - mountPath: /flink-logs
+ name: flink-logs
+ volumes:
+ - name: flink-volume-hostpath
+ hostPath:
+ path: /tmp
+ type: Directory
+ - name: flink-artifact
+ emptyDir: { }
+ - name: flink-logs
+ emptyDir: { }
+```
+
{{< top >}}
diff --git a/docs/content/docs/deployment/resource-providers/native_kubernetes.md b/docs/content/docs/deployment/resource-providers/native_kubernetes.md
index e0e5320..137369f 100644
--- a/docs/content/docs/deployment/resource-providers/native_kubernetes.md
+++ b/docs/content/docs/deployment/resource-providers/native_kubernetes.md
@@ -324,4 +324,240 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit
Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information.
+### Pod Template
+
+Flink allows users to define the JobManager and TaskManager pods via template files. This allows to support advanced features
+that are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly.
+Use [`kubernetes.pod-template-file`]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file)
+to specify a local file that contains the pod definition. It will be used to initialize the JobManager and TaskManager.
+The main container should be defined with name `flink-main-container`.
+Please refer to the [pod template example](#example-of-pod-template) for more information.
+
+#### Fields Overwritten by Flink
+
+Some fields of the pod template will be overwritten by Flink.
+The mechanism for resolving effective field values can be categorized as follows:
+* **Defined by Flink:** User cannot configure it.
+* **Defined by the user:** User can freely specify this value. Flink framework won't set any additional values and the effective value derives from the config option and the template.
+
+ Precedence order: First an explicit config option value is taken, then the value in pod template and at last the default value of a config option if nothing is specified.
+* **Merged with Flink:** Flink will merge values for a setting with a user defined value (see precedence order for "Defined by the user"). Flink values have precedence in case of same name fields.
+
+Refer to the following tables for the full list of pod fields that will be overwritten.
+All the fields defined in the pod template that are not listed in the tables will be unaffected.
+
+**Pod Metadata**
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 10%">Key</th>
+ <th class="text-left" style="width: 20%">Category</th>
+ <th class="text-left" style="width: 30%">Related Config Options</th>
+ <th class="text-left" style="width: 40%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>name</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>The JobManager pod name will be overwritten with the deployment which is defined by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-cluster-id">kubernetes.cluster-id</a>.
+ The TaskManager pod names will be overwritten with the pattern <code><clusterID>-<attempt>-<index></code> which is generated by Flink ResourceManager.</td>
+ </tr>
+ <tr>
+ <td>namespace</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-namespace">kubernetes.namespace</a></td>
+ <td>Both the JobManager deployment and TaskManager pods will be created in the user specified namespace.</td>
+ </tr>
+ <tr>
+ <td>ownerReferences</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>The owner reference of JobManager and TaskManager pods will always be set to the JobManager deployment.
+ Please use <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-owner-reference">kubernetes.jobmanager.owner.reference</a> to control when the deployment is deleted.</td>
+ </tr>
+ <tr>
+ <td>annotations</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-annotations">kubernetes.jobmanager.annotations</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-annotations">kubernetes.taskmanager.annotations</a></td>
+ <td>Flink will add additional annotations specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>labels</td>
+ <td>Merged with Flink</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-labels">kubernetes.jobmanager.labels</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-labels">kubernetes.taskmanager.labels</a></td>
+ <td>Flink will add some internal labels to the user defined values.</td>
+ </tr>
+ </tbody>
+</table>
+
+**Pod Spec**
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 10%">Key</th>
+ <th class="text-left" style="width: 20%">Category</th>
+ <th class="text-left" style="width: 30%">Related Config Options</th>
+ <th class="text-left" style="width: 40%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>imagePullSecrets</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-secrets">kubernetes.container.image.pull-secrets</a></td>
+ <td>Flink will add additional pull secrets specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>nodeSelector</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-node-selector">kubernetes.jobmanager.node-selector</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-node-selector">kubernetes.taskmanager.node-selector</a></td>
+ <td>Flink will add additional node selectors specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>tolerations</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-tolerations">kubernetes.jobmanager.tolerations</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-tolerations">kubernetes.taskmanager.tolerations</a></td>
+ <td>Flink will add additional tolerations specified by the Flink configuration options.</td>
+ </tr>
+ <tr>
+ <td>restartPolicy</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>"always" for JobManager pod and "never" for TaskManager pod.
+ <br>
+ The JobManager pod will always be restarted by deployment. And the TaskManager pod should not be restarted.</td>
+ </tr>
+ <tr>
+ <td>serviceAccount</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-service-account">kubernetes.service-account</a></td>
+ <td>The JobManager and TaskManager pods will be created with the user defined service account.</td>
+ </tr>
+ <tr>
+ <td>volumes</td>
+ <td>Merged with Flink</td>
+ <td></td>
+ <td>Flink will add some internal ConfigMap volumes(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td>
+ </tr>
+ </tbody>
+</table>
+
+**Main Container Spec**
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 10%">Key</th>
+ <th class="text-left" style="width: 20%">Category</th>
+ <th class="text-left" style="width: 30%">Related Config Options</th>
+ <th class="text-left" style="width: 40%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>env</td>
+ <td>Merged with Flink</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.master.env.{ENV_NAME}</a>
+ <a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.taskmanager.env.{ENV_NAME}</a></td>
+ <td>Flink will add some internal environment variables to the user defined values.</td>
+ </tr>
+ <tr>
+ <td>image</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image">kubernetes.container.image</a></td>
+ <td>The container image will be resolved with respect to the defined precedence order for user defined values.</td>
+ </tr>
+ <tr>
+ <td>imagePullPolicy</td>
+ <td>Defined by the user</td>
+ <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-policy">kubernetes.container.image.pull-policy</a></td>
+ <td>The container image pull policy will be resolved with respect to the defined precedence order for user defined values.</td>
+ </tr>
+ <tr>
+ <td>name</td>
+ <td>Defined by Flink</td>
+ <td></td>
+ <td>The container name will be overwritten by Flink with "flink-main-container".</td>
+ </tr>
+ <tr>
+ <td>resources</td>
+ <td>Defined by the user</td>
+ <td>Memory: <br>
+ <a href="{{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size">jobmanager.memory.process.size</a>
+ <a href="{{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size">taskmanager.memory.process.size</a>
+ <br>
+ CPU: <br>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-cpu">kubernetes.jobmanager.cpu</a>
+ <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-cpu">kubernetes.taskmanager.cpu</a></td>
+ <td>The memory and cpu resources(including requests and limits) will be overwritten by Flink configuration options. All other resources(e.g. ephemeral-storage) will be retained.</td>
+ </tr>
+ <tr>
+ <td>containerPorts</td>
+ <td>Merged with Flink</td>
+ <td></td>
+ <td>Flink will add some internal container ports(e.g. rest, jobmanager-rpc, blob, taskmanager-rpc).</td>
+ </tr>
+ <tr>
+ <td>volumeMounts</td>
+ <td>Merged with Flink</td>
+ <td></td>
+ <td>Flink will add some internal volume mounts(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td>
+ </tr>
+ </tbody>
+</table>
+
+#### Example of Pod Template
+`pod-template.yaml`
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+ name: jobmanager-pod-template
+spec:
+ initContainers:
+ - name: artifacts-fetcher
+ image: artifacts-fetcher:latest
+ # Use wget or other tools to get user jars from remote storage
+ command: [ 'wget', 'https://path/of/StateMachineExample.jar', '-O', '/flink-artifact/myjob.jar' ]
+ volumeMounts:
+ - mountPath: /flink-artifact
+ name: flink-artifact
+ containers:
+ # Do not change the main container name
+ - name: flink-main-container
+ resources:
+ requests:
+ ephemeral-storage: 2048Mi
+ limits:
+ ephemeral-storage: 2048Mi
+ volumeMounts:
+ - mountPath: /opt/flink/volumes/hostpath
+ name: flink-volume-hostpath
+ - mountPath: /opt/flink/artifacts
+ name: flink-artifact
+ - mountPath: /opt/flink/log
+ name: flink-logs
+ # Use sidecar container to push logs to remote storage or do some other debugging things
+ - name: sidecar-log-collector
+ image: sidecar-log-collector:latest
+ command: [ 'command-to-upload', '/remote/path/of/flink-logs/' ]
+ volumeMounts:
+ - mountPath: /flink-logs
+ name: flink-logs
+ volumes:
+ - name: flink-volume-hostpath
+ hostPath:
+ path: /tmp
+ type: Directory
+ - name: flink-artifact
+ emptyDir: { }
+ - name: flink-logs
+ emptyDir: { }
+```
+
{{< top >}}