You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/10/11 13:54:31 UTC
[GitHub] [dolphinscheduler] caishunfeng opened a new issue #6499: [Feature][WorkerServer] Tell master to dispatch slowly task when worker is busy
caishunfeng opened a new issue #6499:
URL: https://github.com/apache/dolphinscheduler/issues/6499
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
Worker do ack when it accept a task, but not execute immediately when it's busy, and you can found that the task may be run more than 20s even just a easy echo shell.
<br>
So I add some log at WorkerManagerThread.run, which show that the thread pool queue accumulation at busy time.
<br>
I think if worker is overload, it should tell master to dispatch slowly by more real-time communication.
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #6499: [Feature][WorkerServer] Tell master to dispatch slowly task when worker is busy
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #6499:
URL: https://github.com/apache/dolphinscheduler/issues/6499#issuecomment-940612009
Yes, that's a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6499: [Feature][WorkerServer] Tell master to dispatch slowly task when worker is busy
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6499:
URL: https://github.com/apache/dolphinscheduler/issues/6499#issuecomment-940054053
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] CalvinKirs closed issue #6499: [Feature][WorkerServer] Tell master to dispatch slowly task when worker is busy
Posted by GitBox <gi...@apache.org>.
CalvinKirs closed issue #6499:
URL: https://github.com/apache/dolphinscheduler/issues/6499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] CalvinKirs closed issue #6499: [Feature][WorkerServer] Tell master to dispatch slowly task when worker is busy
Posted by GitBox <gi...@apache.org>.
CalvinKirs closed issue #6499:
URL: https://github.com/apache/dolphinscheduler/issues/6499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on issue #6499: [Feature][WorkerServer] Tell master to dispatch slowly task when worker is busy
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #6499:
URL: https://github.com/apache/dolphinscheduler/issues/6499#issuecomment-940607915
Currently, when the worker's cpu is too high or memory is too low, master will not dispatch a task to it.
Maybe we can add the number of tasks being performed by a worker to the worker metric, consider this metric when selecting workers, here is a related issue. #5196
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org