You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/17 03:57:24 UTC

[GitHub] [airflow] easontm opened a new issue #20898: Metric scheduler.tasks.running is never updated from 0

easontm opened a new issue #20898:
URL: https://github.com/apache/airflow/issues/20898


   ### Apache Airflow version
   
   2.2.3 (latest released)
   
   ### What happened
   
   The metric `scheduler.tasks.running` is populated from the variable `num_tasks_in_executor` in scheduler_job.py. However, `num_tasks_in_executor` is initialized to `0` and then never updated again.
   
   Function `_executable_task_instances_to_queued`:
   https://github.com/apache/airflow/blob/c59001d79facf7e472e0581ac8a538c25eebfda7/airflow/jobs/scheduler_job.py#L328
   
   https://github.com/apache/airflow/blob/c59001d79facf7e472e0581ac8a538c25eebfda7/airflow/jobs/scheduler_job.py#L457
   
   I considered just opening a PR modifying that value with the rest of these "counter" modifications:
   https://github.com/apache/airflow/blob/c59001d79facf7e472e0581ac8a538c25eebfda7/airflow/jobs/scheduler_job.py#L449-L452
   
   However, I'm not sure if this is the best course of action because we don't know if the tasks is successfully added until the the list of executable TIs is returned within `_critical_section_execute_task_instances`, and those TIs are attempted to be queued in `_enqueue_task_instances_with_queued_state`. 
   
   https://github.com/apache/airflow/blob/c59001d79facf7e472e0581ac8a538c25eebfda7/airflow/jobs/scheduler_job.py#L534-L540
   
   https://github.com/apache/airflow/blob/c59001d79facf7e472e0581ac8a538c25eebfda7/airflow/executors/base_executor.py#L73-L85
   
   ### What you expected to happen
   
   _No response_
   
   ### How to reproduce
   
   _No response_
   
   ### Operating System
   
   Ubuntu 19.10
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #20898: Metric scheduler.tasks.running is never updated from 0

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20898:
URL: https://github.com/apache/airflow/issues/20898#issuecomment-1026925042


   Yeah. Confirmed it does look suspiciously 0-always metrics :), Would you maube like to attempt to fix that one @easontm ? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #20898: Metric scheduler.tasks.running is never updated from 0

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20898:
URL: https://github.com/apache/airflow/issues/20898#issuecomment-1026926238


   Opening PR and discussing there, might be the best way to get to the right solution as it will drag attention of those who are closer to the code. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] easontm commented on issue #20898: Metric scheduler.tasks.running is never updated from 0

Posted by GitBox <gi...@apache.org>.
easontm commented on issue #20898:
URL: https://github.com/apache/airflow/issues/20898#issuecomment-1027528597


   I won't be able to do it right away because of other obligations, but I'll see what I can do in a few weeks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org