You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/02/24 09:07:29 UTC
[GitHub] [dolphinscheduler-website] ronyang1985 opened a new pull request #702: [Docs] add doc of EMR Task plugin
ronyang1985 opened a new pull request #702:
URL: https://github.com/apache/dolphinscheduler-website/pull/702
doc for this pr https://github.com/apache/dolphinscheduler/pull/8503
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #702: [doc] Add task plugin doc of EMR
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #702:
URL: https://github.com/apache/dolphinscheduler-website/pull/702#issuecomment-1050536623
Thanks @ronyang1985
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on a change in pull request #702: [Docs] add doc of EMR Task plugin
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on a change in pull request #702:
URL: https://github.com/apache/dolphinscheduler-website/pull/702#discussion_r814423359
##########
File path: docs/zh-cn/dev/user_doc/guide/task/emr.md
##########
@@ -0,0 +1,58 @@
+# Amazon EMR
+
+## 综述
+
+Amazon EMR任务类型，用于在AWS上创建EMR集群并执行计算任务。 后台使用[aws-java-sdk](https://aws.amazon.com/cn/sdk-for-java/) 将json参数转换为[RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html) 对象，提交到AWS
+
+## 任务参数
+- 节点名称：一个工作流定义中的节点名称是唯一的。
+- 运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。
+- 描述信息：描述该节点的功能。
+- 任务优先级：worker线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。
+- Worker分组：任务分配给worker组的机器机执行，选择Default，会随机选择一台worker机执行。
+- 失败重试次数：任务失败重新提交的次数，支持下拉和手填。
+- 失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。
+- 超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.
+- json: [RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html) 对象对应的json，详细json定义参见 [API_RunJobFlow_Examples](https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples)
+
+## json参数样例
+```json·
Review comment:
```suggestion
```json
```
##########
File path: docs/en-us/dev/user_doc/guide/task/emr.md
##########
@@ -0,0 +1,59 @@
+# Amazon EMR
+
+## Overview
+
+Amazon EMR task type, for creating EMR clusters on AWS and performing computing tasks. using [aws-java-sdk](https://aws.amazon.com/cn/sdk-for-java/) in the background, create [RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html) object from json，then submit it to AWS.
+
+## Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail.
+- json: The json corresponding to the [RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html) object，can also refer to [API_RunJobFlow_Examples](https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples)
+
+## json example
+```json·
Review comment:
```suggestion
```json
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #702: [Docs] add doc of EMR Task plugin
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #702:
URL: https://github.com/apache/dolphinscheduler-website/pull/702#issuecomment-1050519878
It seem we have some problem with this PR, CI never end and I could not stop it. I will close and reopen it later to trigger CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie closed pull request #702: [Docs] add doc of EMR Task plugin
Posted by GitBox <gi...@apache.org>.
zhongjiajie closed pull request #702:
URL: https://github.com/apache/dolphinscheduler-website/pull/702
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie merged pull request #702: [doc] Add task plugin doc of EMR
Posted by GitBox <gi...@apache.org>.
zhongjiajie merged pull request #702:
URL: https://github.com/apache/dolphinscheduler-website/pull/702
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org