You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/12/21 03:30:59 UTC
[GitHub] [dolphinscheduler] SbloodyS opened a new issue #7516: [Feature][Master] Optimize master log output
SbloodyS opened a new issue #7516:
URL: https://github.com/apache/dolphinscheduler/issues/7516
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
Master would output too much log when worker is busy or worker thread are running out.
It would generate 2GB's log file every one hour in my production environment.It takes up a lot of hard disk space.
```
[ERROR] 2021-12-21 00:05:13.859 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257857, bodyLen=2481] due to no suitable worker, current task needs worker group master2 to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257857, bodyLen=2481] due to no suitable worker, current task needs worker group master2 to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-12-21 00:05:13.860 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257858, bodyLen=2502] due to no suitable worker, current task needs worker group master2 to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257858, bodyLen=2502] due to no suitable worker, current task needs worker group master2 to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-12-21 00:05:13.860 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257859, bodyLen=2368] due to no suitable worker, current task needs worker group master2 to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257859, bodyLen=2368] due to no suitable worker, current task needs worker group master2 to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-12-21 00:05:13.861 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257860, bodyLen=2481] due to no suitable worker, current task needs worker group master2 to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=257860, bodyLen=2481] due to no suitable worker, current task needs worker group master2 to execute
at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[WARN] 2021-12-21 00:05:12.008 org.apache.dolphinscheduler.server.master.dispatch.host.LowerWeightHostManager:[166] - worker 172.28.2.3:12346 is busy, current waiting task count 69 is large than worker thread count 15
```
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #7516: [Feature][Master] Optimize master log output
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #7516:
URL: https://github.com/apache/dolphinscheduler/issues/7516#issuecomment-1000622049
> @SbloodyS Good idea! What is your suggestion about this?
So far, I can only control by limiting the size and number of log files. There is no better solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo commented on issue #7516: [Feature][Master] Optimize master log output
Posted by GitBox <gi...@apache.org>.
lenboo commented on issue #7516:
URL: https://github.com/apache/dolphinscheduler/issues/7516#issuecomment-998559944
@SbloodyS
Good idea!
What is your suggestion about this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] geosmart edited a comment on issue #7516: [Feature][Master] Optimize master log output
Posted by GitBox <gi...@apache.org>.
geosmart edited a comment on issue #7516:
URL: https://github.com/apache/dolphinscheduler/issues/7516#issuecomment-1004724578
what about add a busy indicator to decrease the log print
like this : the log only print 1 time in 60 second
```java
var interval = 0
if (worker.isBusy()){
if(interval==60){
logger.info(''xxx'')
interval = 0
}
interval++
sleep(1s)
}else{
//
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #7516: [Feature][Master] Optimize master log output
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #7516:
URL: https://github.com/apache/dolphinscheduler/issues/7516#issuecomment-998442273
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] geosmart commented on issue #7516: [Feature][Master] Optimize master log output
Posted by GitBox <gi...@apache.org>.
geosmart commented on issue #7516:
URL: https://github.com/apache/dolphinscheduler/issues/7516#issuecomment-1004724578
what about add a busy indicator to decrease the log print
like this : the log only print 1 time in 60 second
```java
var interval = 0
if (worker.isBusy()){
if(interval==60){
logger.info(''xxx'')
interval = 0
}
interval++
sleep(1s)
} else{
//
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org