You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/08/26 02:52:11 UTC
[GitHub] [incubator-dolphinscheduler] pipiwukong opened a new issue #3595: [Question][server] Execution of the subprocess is very inefficient。（子流程的执行效率非常低）
pipiwukong opened a new issue #3595:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3595
*For better global communication, please give priority to using English description, thx! *
*Please review https://dolphinscheduler.apache.org/en-us/docs/development/issue.html when describe an issue.*
**Describe the question**
If a process has some subprocesses. These subprocesses can't be excuted in parallel efficiently.
I have finished a test in two situations.
case1： Ten processes run together. Tasks of ten processes can be excuted in parallel. The task concurrency can reach more the 80.
case2：Create a new processes to excute ten processes as subprocesses. Tasks of ten subprocesses can't be excuted in parallel. Just the tasks in one subprocess can be excuted in parallel. The task concurrency is lower the 20.
How to improve the performance of the subprocess？
**Which version of DolphinScheduler:**
-[1.3.1]
**Additional context**
Add any other context about the problem here.
**Requirement or improvement**
Analysing the DAGs of the subprocesses together to find the tasks can be excuted. Then excute these tasks in parallel.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] KnightChess edited a comment on issue #3595: [Bug][server] Execution of the subprocess is very inefficient。（子流程的执行效率非常低）
Posted by GitBox <gi...@apache.org>.
KnightChess edited a comment on issue #3595:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3595#issuecomment-715916121
a process instance correspond a masterExecThread，a masterExecThread only can start up 20 taskExecThread to excute task in parallel default，if you want improve it，modify parameter master.exec.task.num=20，80 or more in master.properties
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] KnightChess commented on issue #3595: [Bug][server] Execution of the subprocess is very inefficient。（子流程的执行效率非常低）
Posted by GitBox <gi...@apache.org>.
KnightChess commented on issue #3595:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3595#issuecomment-715916121
a process instance correspond a masterExecThread，a masterExecThread only can start up 20 taskExecThread to excute task in parallel，if you want improve it，modify parameter master.exec.task.num=20，80 or more in master.properties
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen closed issue #3595: [Bug][server] Execution of the subprocess is very inefficient。（子流程的执行效率非常低）
Posted by GitBox <gi...@apache.org>.
xingchun-chen closed issue #3595:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3595
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org