[GitHub] [dolphinscheduler] caishunfeng commented on issue #9401: [Bug] [Master] Master frequent print logs

[GitBox <gi...@apache.org>]

caishunfeng commented on issue #9401:
URL: https://github.com/apache/dolphinscheduler/issues/9401#issuecomment-1099794743

   > > Through communication, I learned the current situation: The task itself must be executed successfully, so it needs to be retried continuously. If there is a problem when pushing it to `worker`, the log must also be output.
   > > But I think outputting logs actually means some kind of feedback or recording problems. I speculate that adding status to `task` itself may solve the problem. For example, `waiting` and `pending`, `waiting` means that the `task` instance is newly created, and `pending` means that the pre execution logic is satisfied and attempts to push it to `worker`.
   > > I suggest updating from `waiting` to `pending` and pushing the task to `worker` when preparing to push the task, so that users can realize that the task has met the execution conditions. If the `task` in `waiting` status fails to be pushed to `worker` for the first time, the error log can be printed and updated to `pending` status. After that, try again and again. If the `task` in the `pending` state fails to push, the log will not be output. When it succeeds, switch to the current execution state and output the corresponding success log.
   > > Of course, how the underlying method of this logic call controls the log needs to be discussed.
   > 
   > I think i can make this simple to understand. The solution is to add the status of task instances to replace the log output.
   > 
   > 1. Add `waiting suitable worker` state to represent no suitable worker log output.
   > 2. Add `waiing for the worker's load to decrease` state to represent worker in high load.
   > 3. Add `stop due to the running task was deleted` state to represent the running task was deleted.
   > 
   > After adding these status to the task instance. We may delete the irrelevant log to prevent frequent log printing. So that the user can understand task instance state instead of locate the problem by querying the logs. And this solution is also applicable to other places where logs are frequently printed.
   > 
   > WDYT? @caishunfeng @lenboo @JinyLeeChina
   > 
   > Related issue: #7516
   
   Good idea, but I think just add `waitting suitable worker` state for dispatch is OK, because `fail to execute : ... due to no suitable worker, current task needs worker group ... to execute` print the most.  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org