You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/07/09 03:29:30 UTC

[GitHub] [incubator-dolphinscheduler] Jops-data opened a new issue #3168: How to control workflow control concurrent submission task node

Jops-data opened a new issue #3168:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3168


   *For better global communication, please give priority to using English description, thx! *
   **Describe the question**
   If a workflow does not provide up-down dependency to connect task nodes, all tasks will be submitted at one time when the workflow is executed, which will cause the CPU of the machine to burst. Now I'd like to ask and then control the number of node submissions for the workflow.
   
   **Which version of DolphinScheduler:**
   1.2.1
   
   **Requirement or improvement
   I hope that the maximum number of submission nodes can be set in a workflow to control the number of submitted tasks, rather than through the dependency relationship between nodes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] BoYiZhang commented on issue #3168: How to control workflow control concurrent submission task node

Posted by GitBox <gi...@apache.org>.
BoYiZhang commented on issue #3168:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3168#issuecomment-655951135


   You can configure [ master.properties ]
   such as:
   
   # master execute thread num
   #master.exec.threads=100
   
   # master execute task number in parallel
   #master.exec.task.num=20
   
   
   # master dispatch task number
   #master.dispatch.task.num = 3
   
   
   # only less than cpu avg load, master server can work.  default value -1 : the number of cpu cores * 2
   #master.max.cpuload.avg=-1
   
   # only larger than reserved memory, master server can work. default value : physical memory * 1/10, unit is G.
   #master.reserved.memory=0.3
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] BoYiZhang edited a comment on issue #3168: How to control workflow control concurrent submission task node

Posted by GitBox <gi...@apache.org>.
BoYiZhang edited a comment on issue #3168:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3168#issuecomment-655951135


   You can configure [ master.properties ]
   such as:
   ````
   # master execute thread num
   #master.exec.threads=100
   
   # master execute task number in parallel
   #master.exec.task.num=20
   
   
   # master dispatch task number
   #master.dispatch.task.num = 3
   
   
   # only less than cpu avg load, master server can work.  default value -1 : the number of cpu cores * 2
   #master.max.cpuload.avg=-1
   
   # only larger than reserved memory, master server can work. default value : physical memory * 1/10, unit is G.
   #master.reserved.memory=0.3
   ````
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] Jops-data commented on issue #3168: How to control workflow control concurrent submission task node

Posted by GitBox <gi...@apache.org>.
Jops-data commented on issue #3168:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3168#issuecomment-658016405


   Yes, this idea can be solved！


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #3168: How to control workflow control concurrent submission task node

Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #3168:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3168#issuecomment-657990266


   did you mean this feature idea? https://github.com/apache/incubator-dolphinscheduler/issues/268


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] Jops-data closed issue #3168: How to control workflow control concurrent submission task node

Posted by GitBox <gi...@apache.org>.
Jops-data closed issue #3168:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3168


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org