You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/03/04 07:33:58 UTC
[GitHub] [dolphinscheduler-website] EricGao888 opened a new pull request #713: [Feature-8023][Document] Add example and notice about task type Python
EricGao888 opened a new pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713
- Based on the [shell task example](https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/task/shell.html), this pr adds example and notice about task type Python
- This pr closes: #8023
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on a change in pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on a change in pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#discussion_r840492545
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
Review comment:
Fixed in the latest commit.
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
Review comment:
Fixed in the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1081347756
> docs looking good, but could you change the screenshot to latest in branch dev? We change our UI recently which you can see https://dolphinscheduler.apache.org/en-us/docs/dev/user_doc/guide/task/flink.html as example
Sure, the new UI looks cool!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on a change in pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on a change in pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#discussion_r839300397
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
Review comment:
I find that we have some docs overwrite in other PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1059285292
will task a look tomorrow, is midnight now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1060139780
> Looking good! thanks @EricGao888. FYI you just need to change `dev` and latest(for now is`2.0.3`) doc in both EN and ZH doc. we could jus ignore previous verisons
Sure, thx for the comment! Maybe we can adopt jinjava or other template lib to generate doc in the feture? I think that may save a lot of work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1085750071
Thanks @Tianqi-Dotes for reviewing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] Tianqi-Dotes commented on a change in pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
Tianqi-Dotes commented on a change in pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#discussion_r840433761
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
Review comment:
better
a temporary python script is generated,
change to
will generate a temporary python script,
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
Review comment:
better
`Python Task` is used to create
change to
Use `Python Task` to create
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
Review comment:
When the worker execute
change to
When the worker executes
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
Review comment:
', and the Linux user with the same name as the tenant executes the script.'
should be
', and executes the script by the Linux user with the same name as the tenant .'
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
Review comment:
When the number of worker threads is insufficient, execute in the order of priority from high to low, and tasks with the same priority will execute in a first-in first-out order.
should be better
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
Review comment:
Assign tasks to the machines of the worker group to execute. If `Default` is selected, randomly select a worker machine for execution.
should be better
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Environment Name: Configure the environment name in which to run the script.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail.
+- Script: PYTHON program developed by users.
+- Resource: Refers to the list of resource files that need to be called in the script, and the files uploaded or created by the resource center-file management.
+- Custom parameters: It is a user-defined parameter that is part of PYTHON, which will replace the content with \${variable} in the script.
Review comment:
It is a user-defined parameter that is part of PYTHON
to
It is the user-defined parameters of Python,
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Environment Name: Configure the environment name in which to run the script.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail.
Review comment:
will be sent
to
will send
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Environment Name: Configure the environment name in which to run the script.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail.
+- Script: PYTHON program developed by users.
+- Resource: Refers to the list of resource files that need to be called in the script, and the files uploaded or created by the resource center-file management.
+- Custom parameters: It is a user-defined parameter that is part of PYTHON, which will replace the content with \${variable} in the script.
+
+## Task Example
+
+### Simply Print
+
+This example is a sample print task which only print one line in the log file, including the content
+"This is a demo of python task". If your task only run one or two python command, you could add task based on this example.
Review comment:
This is different from the original meaning.
should be
This example simulates a common task that runs by a simple command. The example is to print one line in the log file, as shown in the following figure:
"This is a demo of python task".
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
Review comment:
describe -> Describe
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Environment Name: Configure the environment name in which to run the script.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail.
+- Script: PYTHON program developed by users.
Review comment:
Python program developed by the user.
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
Review comment:
drawing board
to
canvas
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Environment Name: Configure the environment name in which to run the script.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
Review comment:
better
The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
to
The failure task resubmitting times.
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
+- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
+- Drag <img src="/img/tasks/icons/python.png" width="15"/> from the toolbar to the drawing board.
+
+## Task Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Environment Name: Configure the environment name in which to run the script.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail.
+- Script: PYTHON program developed by users.
+- Resource: Refers to the list of resource files that need to be called in the script, and the files uploaded or created by the resource center-file management.
+- Custom parameters: It is a user-defined parameter that is part of PYTHON, which will replace the content with \${variable} in the script.
+
+## Task Example
+
+### Simply Print
+
+This example is a sample print task which only print one line in the log file, including the content
+"This is a demo of python task". If your task only run one or two python command, you could add task based on this example.
+
+
+
+```python
+print("This is a demo of python task")
+```
+
+### Custom Parameters
+
+This example is a sample custom parameter task which could reuse existing as template, or for dynamic task. First of all,
+we should declare a custom parameter named "param_key", with the value as "param_val". Then we using keyword "${param_key}"
+to using the parameter we just declared. After this example is being run, we would see "param_val" print in the log
Review comment:
better change to
This example simulates a custom parameter task. We use parameters for reusing existing tasks as template or coping with the dynamic task. In this case,
we declare a custom parameter named "param_key", with the value "param_val". Then we use `echo` to print the parameter "${param_key}" we just declared.
After running this example, we would see "param_val" print in the log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1083987449
I approved the CI running just now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1060172628
BTW, we plan to move our docs into main repo. After that we only need to change dev version docs. (in most cases)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 edited a comment on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 edited a comment on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1060139780
> Looking good! thanks @EricGao888. FYI you just need to change `dev` and latest(for now is`2.0.3`) doc in both EN and ZH doc. we could jus ignore previous verisons
Sure, thx for the comment! BTW maybe we can adopt jinjava or other template lib to generate doc in the feture? I think that may save a lot of work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on a change in pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on a change in pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#discussion_r840254198
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
Review comment:
@zhongjiajie @Tianqi-Dotes Is there anything else I could do here? I mean the docs overwrite stuff, not sure what it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1058910841
> * Based on the [shell task example](https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/task/shell.html), this pr adds example and notice about task type Python
> * This pr closes: #8023
@zhongjiajie FYi, this pr is WIP. I only fix the python task examples in English dev version in this commit. If things look good, I will fix python task examples in other versions of docs in the following commits. Thx : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1085786947
> Hi Mr. Gao, please try not to use passive patterns in technology writing. Thanks for the contribution. Yours
@Tianqi-Dotes Hi Tianqi, thx for helping me review the docs in such detail. I've fixed the related language issues and resolved the comments : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on a change in pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on a change in pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#discussion_r839298718
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
+a temporary python script is generated, and the Linux user with the same name as the tenant executes the script.
-<p align="center">
- <img src="/img/python-en.png" width="80%" />
- </p>
+## Create Task
-- Script: Python program developed by the user.
-- Environment Name: Specific Python interpreter path for running the script. If you need to use Python **virtualenv**, you should create multiply environments for each **virtualenv**.
-- Resources: Refers to the list of resource files that need to be called in the script.
-- User-defined parameter: It is a user-defined local parameter of Python, and will replace the content with `${variable}` in the script.
-- Note: If you import the python file under the resource directory tree, you need to add the `__init__.py` file.
Review comment:
BTW, @Tianqi-Dotes could you make a double check about our dev EN document change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1060171743
Good point, yeah, we already have a plan in https://github.com/apache/dolphinscheduler/issues/6535, section `DRY`. But our community do not have time to do it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on a change in pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on a change in pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#discussion_r840492423
##########
File path: docs/en-us/dev/user_doc/guide/task/python.md
##########
@@ -1,15 +1,55 @@
# Python Node
-- Using python nodes, you can directly execute python scripts. For python nodes, workers use `python **` to submit tasks.
+## Overview
-> Drag from the toolbar  task node to the canvas, as shown in the following figure:
+`Python Task` is used to create a python-type task and execute python scripts. When the worker execute `Python Task`,
Review comment:
Fixed in the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1085927391
> I will merge it when it finishes the CI, and then will do the doc migration
@zhongjiajie Cool, I'm looking forward to the doc migration. After migration, it seems easier for developers to contribute to dolphin docs and I'd like to keep contributing to these docs : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1081326428
> Looking good! thanks @EricGao888. FYI you just need to change `dev` and latest(for now is`2.0.3`) doc in both EN and ZH doc. we could jus ignore previous verisons
@zhongjiajie I've changed the `dev` and latest(2.0.5) doc in both EN and ZH versions. PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] EricGao888 commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1083254639
> docs looking good, but could you change the screenshot to latest in branch dev? We change our UI recently which you can see https://dolphinscheduler.apache.org/en-us/docs/dev/user_doc/guide/task/flink.html as example
Have changed the screenshots in dev directory to match the newest UI : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie commented on pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713#issuecomment-1085896953
I will merge it when it finishes the CI, and then will do the doc migration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-website] zhongjiajie merged pull request #713: [Feature-8023][Document] Add example and notice about task type Python
Posted by GitBox <gi...@apache.org>.
zhongjiajie merged pull request #713:
URL: https://github.com/apache/dolphinscheduler-website/pull/713
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org