You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/05/13 15:43:01 UTC

[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #9425: [Feature][plugin] Add k8s task in task plugin

zhongjiajie commented on code in PR #9425:
URL: https://github.com/apache/dolphinscheduler/pull/9425#discussion_r872528284


##########
docs/docs/en/guide/task/k8s.md:
##########
@@ -0,0 +1,43 @@
+# K8S Node

Review Comment:
   when you add new task, you should also add it to sidebar in https://github.com/apache/dolphinscheduler/blob/dev/docs/configs/docsdev.js#L177



##########
docs/docs/en/guide/task/k8s.md:
##########
@@ -0,0 +1,43 @@
+# K8S Node
+
+## Overview
+
+K8S task type used to execute a batch task. In this task, the worker submits the task by using a k8s client.
+
+## Create Task
+
+- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page.
+- Drag from the toolbar <img src="/img/tasks/icons/k8s.png" width="15"/> to the canvas.
+
+## Task Parameter

Review Comment:
   ```suggestion
   ## Task Parameter
   
   ```



##########
docs/docs/en/guide/task/k8s.md:
##########
@@ -0,0 +1,43 @@
+# K8S Node
+
+## Overview
+
+K8S task type used to execute a batch task. In this task, the worker submits the task by using a k8s client.
+
+## Create Task
+
+- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page.
+- Drag from the toolbar <img src="/img/tasks/icons/k8s.png" width="15"/> to the canvas.
+
+## Task Parameter
+- **Node name**: The node name in a workflow definition is unique.
+- **Run flag**: Identifies whether this node schedules normally, if it does not need to execute, select the `prohibition execution`.
+- **Descriptive information**: Describe the function of the node.
+- **Task priority**: When the number of worker threads is insufficient, execute in the order of priority from high to low, and tasks with the same priority will execute in a first-in first-out order.
+- **Worker grouping**: Assign tasks to the machines of the worker group to execute. If `Default` is selected, randomly select a worker machine for execution.
+- **Environment Name**: Configure the environment name in which to run the task.
+- **Times of failed retry attempts**: The number of times the task failed to resubmit.
+- **Failed retry interval**: The time interval (unit minute) for resubmitting the task after a failed task.
+- **Delayed execution time**: The time (unit minute) that a task delays in execution.
+- **Timeout alarm**: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm email will send and the task execution will fail.
+- **Namespace**::the namespace for running k8s task
+- **Min CPU**:min CPU requirement for running k8s task
+- **Min Memory**:min memory requirement for running k8s task
+- **Image**:the registry url for image 
+- **Custom parameter**: It is a local user-defined parameter for K8S task, these params will pass to container as environment variables.
+- **Predecessor task**: Selecting a predecessor task for the current task, will set the selected predecessor task as upstream of the current task.
+## Task Example
+
+#### Configure the K8S Environment in DolphinScheduler

Review Comment:
   ```suggestion
   ### Configure the K8S Environment in DolphinScheduler
   ```



##########
docs/docs/zh/guide/task/k8s.md:
##########
@@ -0,0 +1,46 @@
+# K8S
+
+## 综述
+
+K8S任务类型,用于在K8S上执行一个短时和批处理的任务。worker最终会通过使用K8S client提交任务。
+
+

Review Comment:
   ```suggestion
   ```



##########
docs/docs/zh/guide/task/k8s.md:
##########
@@ -0,0 +1,46 @@
+# K8S
+
+## 综述
+
+K8S任务类型,用于在K8S上执行一个短时和批处理的任务。worker最终会通过使用K8S client提交任务。
+
+
+## 创建任务
+
+- 点击项目管理-项目名称-工作流定义,点击"创建工作流"按钮,进入DAG编辑页面。
+- 工具栏中拖动 <img src="/img/tasks/icons/k8s.png" width="25"/> 到画板中,选择需要连接的数据源,即可完成创建。
+
+## 任务参数
+
+- 节点名称:设置任务的名称。一个工作流定义中的节点名称是唯一的。
+- 运行标志:标识这个节点是否能正常调度,如果不需要执行,可以打开禁止执行开关。
+- 描述:描述该节点的功能。
+- 任务优先级:worker 线程数不足时,根据优先级从高到低依次执行,优先级一样时根据先进先出原则执行。
+- Worker 分组:任务分配给 worker 组的机器执行,选择 Default 会随机选择一台 worker 机执行。
+- 环境名称:配置运行任务的环境。
+- 失败重试次数:任务失败重新提交的次数。
+- 失败重试间隔:任务失败重新提交任务的时间间隔,以分为单位。
+- 延迟执行时间:任务延迟执行的时间,以分为单位。
+- 超时警告:勾选超时警告、超时失败,当任务超过“超时时长”后,会发送告警邮件并且任务执行失败。
+- 命名空间:选择K8S集群上存在的命名空间
+- 最小CPU:任务在K8S上运行所需的最小CPU
+- 最小内存:任务在K8S上运行所需的最小内存
+- 镜像:镜像地址
+- 自定义参数:K8S任务局部的用户自定义参数,自定义参数最终会通过环境变量形式存在于容器中,提供给K8S任务使用
+- 前置任务:在当前K8S任务之前需要执行的任务
+
+## 任务样例
+
+### 在 DolphinScheduler 中配置 K8S 集群环境
+
+若生产环境中要是使用到 K8s 任务类型,则需要预先配置好所需的K8S集群环境
+
+### 配置 K8S 任务节点
+
+根据上述参数说明,配置所需的内容即可。
+
+![K8S](/img/tasks/demo/k8s-task-zh.png)

Review Comment:
   please only use English img in our document, you could use English img to Chinese doc too. BTW, change the img file name to `kubernetes-task.png`



##########
docs/docs/en/guide/task/k8s.md:
##########
@@ -0,0 +1,43 @@
+# K8S Node

Review Comment:
   BTW, please use the keyword `kubernetes` instead of `K8S` in the document, more formal



##########
docs/docs/en/guide/task/k8s.md:
##########
@@ -0,0 +1,43 @@
+# K8S Node
+
+## Overview
+
+K8S task type used to execute a batch task. In this task, the worker submits the task by using a k8s client.
+
+## Create Task
+
+- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page.
+- Drag from the toolbar <img src="/img/tasks/icons/k8s.png" width="15"/> to the canvas.
+
+## Task Parameter
+- **Node name**: The node name in a workflow definition is unique.
+- **Run flag**: Identifies whether this node schedules normally, if it does not need to execute, select the `prohibition execution`.
+- **Descriptive information**: Describe the function of the node.
+- **Task priority**: When the number of worker threads is insufficient, execute in the order of priority from high to low, and tasks with the same priority will execute in a first-in first-out order.
+- **Worker grouping**: Assign tasks to the machines of the worker group to execute. If `Default` is selected, randomly select a worker machine for execution.
+- **Environment Name**: Configure the environment name in which to run the task.
+- **Times of failed retry attempts**: The number of times the task failed to resubmit.
+- **Failed retry interval**: The time interval (unit minute) for resubmitting the task after a failed task.
+- **Delayed execution time**: The time (unit minute) that a task delays in execution.
+- **Timeout alarm**: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm email will send and the task execution will fail.
+- **Namespace**::the namespace for running k8s task
+- **Min CPU**:min CPU requirement for running k8s task
+- **Min Memory**:min memory requirement for running k8s task
+- **Image**:the registry url for image 
+- **Custom parameter**: It is a local user-defined parameter for K8S task, these params will pass to container as environment variables.
+- **Predecessor task**: Selecting a predecessor task for the current task, will set the selected predecessor task as upstream of the current task.
+## Task Example
+
+#### Configure the K8S Environment in DolphinScheduler
+
+If you are using the K8S task type in a production environment, the K8S cluster environment is required.
+
+#### Configure K8S Nodes

Review Comment:
   ```suggestion
   ### Configure K8S Nodes
   ```



##########
docs/docs/en/guide/task/k8s.md:
##########
@@ -0,0 +1,43 @@
+# K8S Node

Review Comment:
   Also change the file name too, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org