You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/01/27 04:05:34 UTC
[GitHub] [dolphinscheduler] AntAndfish opened a new issue #8213: [Bug] [dolphinscheduler-server] task run error when worker group name contains uppercase letters
AntAndfish opened a new issue #8213:
URL: https://github.com/apache/dolphinscheduler/issues/8213
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
worker group名称存在大写的时候，运行任务报错；错误如下
_[INFO] 2022-01-27 08:00:00.389 TaskLogLogger:[134] - set work flow 299 task 463 running
[INFO] 2022-01-27 08:00:00.390 TaskLogLogger:[104] - work flow 299 task 463, sub work flow: 301 state: running
[ERROR] 2022-01-27 08:00:01.081 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[144] - ExecuteException dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511226, bodyLen=3375] due to no suitable worker, current task needs worker group test to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511226, bodyLen=3375] due to no suitable worker, current task needs worker group test to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:138)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:101)
[ERROR] 2022-01-27 08:00:01.082 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[144] - ExecuteException dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511227, bodyLen=3106] due to no suitable worker, current task needs worker group Tenant_2_DS资源组3 to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511227, bodyLen=3106] due to no suitable worker, current task needs worker group Tenant_2_DS资源组3 to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:138)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:101)
[WARN] 2022-01-27 08:00:01.360 org.apache.dolphinscheduler.server.master.dispatch.host.LowerWeightHostManager:[153] - worker 192.168.92.31:1234 in work group test have not received the heartbeat
[WARN] 2022-01-27 08:00:02.361 org.apache.dolphinscheduler.server.master.dispatch.host.LowerWeightHostManager:[153] - worker 192.168.92.31:1234 in work group test have not received the heartbeat
_
原因：master ServerNodeManager.syncWorkerGroupNodes() 方法，将读取到的worker group名称转为了小写，但是，从任务内读取的workergroup名称仍然是大写，因此任务找不到需要的worker group；
处理：将//workerGroup = workerGroup.toLowerCase(); 注释掉。
` private void syncWorkerGroupNodes(String workerGroup, Collection<String> nodes) {
workerGroupLock.lock();
try {
//workerGroup = workerGroup.toLowerCase();
Set<String> workerNodes = workerGroupNodes.getOrDefault(workerGroup, new HashSet<>());
workerNodes.clear();
workerNodes.addAll(nodes);
workerGroupNodes.put(workerGroup, workerNodes);
} finally {
workerGroupLock.unlock();
}
}
`
### What you expected to happen
当worker group 名称存在大写字符时，任务正常运行
### How to reproduce
创建一个包含大写字符的workergroup 名称，创建任意一个任务并关联到 创建的worker group 运行；
### Anything else
_No response_
### Version
2.0.3
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #8213: [Bug] [dolphinscheduler-server] task run error when worker group name contains uppercase letters
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #8213:
URL: https://github.com/apache/dolphinscheduler/issues/8213#issuecomment-1022827957
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
worker group名称存在大写的时候，运行任务报错；错误如下
_[INFO] 2022-01-27 08:00:00.389 TaskLogLogger:[134] - set work flow 299 task 463 running
[INFO] 2022-01-27 08:00:00.390 TaskLogLogger:[104] - work flow 299 task 463, sub work flow: 301 state: running
[ERROR] 2022-01-27 08:00:01.081 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[144] - ExecuteException dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511226, bodyLen=3375] due to no suitable worker, current task needs worker group test to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511226, bodyLen=3375] due to no suitable worker, current task needs worker group test to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:138)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:101)
[ERROR] 2022-01-27 08:00:01.082 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[144] - ExecuteException dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511227, bodyLen=3106] due to no suitable worker, current task needs worker group Tenant_2_DS资源组3 to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=3511227, bodyLen=3106] due to no suitable worker, current task needs worker group Tenant_2_DS资源组3 to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:138)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:101)
[WARN] 2022-01-27 08:00:01.360 org.apache.dolphinscheduler.server.master.dispatch.host.LowerWeightHostManager:[153] - worker 192.168.92.31:1234 in work group test have not received the heartbeat
[WARN] 2022-01-27 08:00:02.361 org.apache.dolphinscheduler.server.master.dispatch.host.LowerWeightHostManager:[153] - worker 192.168.92.31:1234 in work group test have not received the heartbeat
_
原因：master ServerNodeManager.syncWorkerGroupNodes() 方法，将读取到的worker group名称转为了小写，但是，从任务内读取的workergroup名称仍然是大写，因此任务找不到需要的worker group；
处理：将//workerGroup = workerGroup.toLowerCase(); 注释掉。
` private void syncWorkerGroupNodes(String workerGroup, Collection<String> nodes) {
workerGroupLock.lock();
try {
//workerGroup = workerGroup.toLowerCase();
Set<String> workerNodes = workerGroupNodes.getOrDefault(workerGroup, new HashSet<>());
workerNodes.clear();
workerNodes.addAll(nodes);
workerGroupNodes.put(workerGroup, workerNodes);
} finally {
workerGroupLock.unlock();
}
}
`
### What you expected to happen
当worker group 名称存在大写字符时，任务正常运行
### How to reproduce
创建一个包含大写字符的workergroup 名称，创建任意一个任务并关联到 创建的worker group 运行；
### Anything else
_No response_
### Version
2.0.3
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng closed issue #8213: [Bug] [dolphinscheduler-server] task run error when worker group name contains uppercase letters
Posted by GitBox <gi...@apache.org>.
caishunfeng closed issue #8213:
URL: https://github.com/apache/dolphinscheduler/issues/8213
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] JinyLeeChina commented on issue #8213: [Bug] [dolphinscheduler-server] task run error when worker group name contains uppercase letters
Posted by GitBox <gi...@apache.org>.
JinyLeeChina commented on issue #8213:
URL: https://github.com/apache/dolphinscheduler/issues/8213#issuecomment-1035957014
@AntAndfish，is pr submitted? Please also associate this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #8213: [Bug] [dolphinscheduler-server] task run error when worker group name contains uppercase letters
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #8213:
URL: https://github.com/apache/dolphinscheduler/issues/8213#issuecomment-1042658864
@AntAndfish , sorry, we hope to fix this issue in 2.0.4, so I unassign to you because long time no reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #8213: [Bug] [dolphinscheduler-server] task run error when worker group name contains uppercase letters
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #8213:
URL: https://github.com/apache/dolphinscheduler/issues/8213#issuecomment-1022828091
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #8213: [Bug] [dolphinscheduler-server] task run error when worker group name contains uppercase letters
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #8213:
URL: https://github.com/apache/dolphinscheduler/issues/8213#issuecomment-1031125961
Hi @AntAndfish, it's a good issue with detail, but it's better to describe in English, so that everyone in the community can understand it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org