You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/14 16:53:13 UTC

[GitHub] [airflow] SolbiatiAlessandro opened a new issue #12368: Metrics - Introducing a counter for number of DAGs in "running" state

SolbiatiAlessandro opened a new issue #12368:
URL: https://github.com/apache/airflow/issues/12368

**Description**

Could we introduce a new counter in [Airflow Metrics](https://airflow.apache.org/docs/stable/metrics.html) to track **number of DAGs that are in 'running' state**?

I am aware of the existing `dag_processing.processes` counter that is documented as `Number of currently running DAG parsing processes`. This counter track how many dags are being parsed, not how many are runnings.

I am aware of the existing `executor.queued_tasks` and `executor.running_tasks`, but they are task-wise and not dag-wise.

**Use case / motivation**

Consider this example
![Screenshot 2020-11-14 at 16 42 40](https://user-images.githubusercontent.com/20618047/99152409-7a619780-2699-11eb-9680-bf74cb5adc67.png)

I want to know how many DAGs are in the state of `running` to understand memory consumption issues and to understand how long is my "queue" of DAGs. To my understanding there is no metric currently that I can use to track that.

I propose a counter `executor.running_dags` that would return the number of DAGs in the running state.

Does this make sense? Am I missing something? If folks I agree I would be happy to work on a PR :)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] mik-laj commented on issue #12368: Metrics - Introducing a counter for number of DAGs in "running" state

Posted by GitBox <gi...@apache.org>.

mik-laj commented on issue #12368:
URL: https://github.com/apache/airflow/issues/12368#issuecomment-727273077


   @SolbiatiAlessandro Yes. Please do it. ;-)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #12368: Metrics - Introducing a counter for number of DAGs in "running" state

Posted by GitBox <gi...@apache.org>.

boring-cyborg[bot] commented on issue #12368:
URL: https://github.com/apache/airflow/issues/12368#issuecomment-727234430


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] SolbiatiAlessandro commented on issue #12368: Metrics - Introducing a counter for number of DAGs in "running" state

Posted by GitBox <gi...@apache.org>.

SolbiatiAlessandro commented on issue #12368:
URL: https://github.com/apache/airflow/issues/12368#issuecomment-727271638


   @mik-laj  thanks for the repsonse! I will look into that :) 
   Do you think we should add two lines in this doc page to mention that statsd only track non-database information and you can find more metrics in the database? https://airflow.apache.org/docs/stable/metrics.html


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb commented on issue #12368: Metrics - Introducing a counter for number of DAGs in "running" state

Posted by GitBox <gi...@apache.org>.

ashb commented on issue #12368:
URL: https://github.com/apache/airflow/issues/12368#issuecomment-727638083


   Additionally: we can't (easily) report this as a metric now that running more than on e scheduler is supported.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] mik-laj commented on issue #12368: Metrics - Introducing a counter for number of DAGs in "running" state

Posted by GitBox <gi...@apache.org>.

mik-laj commented on issue #12368:
URL: https://github.com/apache/airflow/issues/12368#issuecomment-727262520


   This information is in the database so you can use:
   https://github.com/robinhood/airflow-prometheus-exporter or other similar exporter
   
   We use statsd metrics only for information that we do not have in the database, i.e. runtime metrics.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] mik-laj edited a comment on issue #12368: Metrics - Introducing a counter for number of DAGs in "running" state

Posted by GitBox <gi...@apache.org>.

mik-laj edited a comment on issue #12368:
URL: https://github.com/apache/airflow/issues/12368#issuecomment-727262520


   This information is in the database so you can use:
   https://github.com/robinhood/airflow-prometheus-exporter or https://github.com/PBWebMedia/airflow-prometheus-exporter or other similar exporter
   
   We use statsd metrics only for information that we do not have in the database, i.e. runtime metrics.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org