You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/28 14:33:39 UTC

[GitHub] [airflow] mjpieters opened a new issue #10629: Several statsd timers record seconds rather than milliseconds

mjpieters opened a new issue #10629:
URL: https://github.com/apache/airflow/issues/10629


   **Apache Airflow version**: 1.10.11
   
   **What happened**:
   
   Statsd timers for 3 categories are reporting values 1000x off as the code passes seconds, rather rather than milliseconds (or a `timedelta()` instance) to [`StatsClient.timing()`](https://statsd.readthedocs.io/en/v3.3/reference.html#StatsClient.timing).
   
   **What you expected to happen**:
   
   The timings should be reported in milliseconds.
   
   **How to reproduce it**:
   
   Enable the `statsd` client and track reporting timings for:
   
   * `dag.loading-duration.*` (emitted from `models/dagbag.py`)
   * `dag.<dag_id>.<task_id>.duration` (emitted from `models/taskinstance.py`)
   * `dag_processing.last_duration.<filename>` (emitted from `utils/dag_processing.py`) and it's deprecated alias `dag_processing.last_runtime.*`
   
   **Anything else we need to know**:
   
   This was reported before, for the `dag_processing.last_*` stats, at https://issues.apache.org/jira/browse/AIRFLOW-6088.
   The accompanying pull request #6682, fixed 2 out of 3 of these, but only for Airflow 2.0.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb closed issue #10629: Several statsd timers record seconds rather than milliseconds

Posted by GitBox <gi...@apache.org>.
ashb closed issue #10629:
URL: https://github.com/apache/airflow/issues/10629


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb edited a comment on issue #10629: Several statsd timers record seconds rather than milliseconds

Posted by GitBox <gi...@apache.org>.
ashb edited a comment on issue #10629:
URL: https://github.com/apache/airflow/issues/10629#issuecomment-733907248


   I think there may be one left -- in DagBag.collect_dags
   
   ```
           durations = (end_dttm - start_dttm).total_seconds()
           Stats.gauge('collect_dags', durations, 1)
   ```
   
   Oh, that's a guage.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb commented on issue #10629: Several statsd timers record seconds rather than milliseconds

Posted by GitBox <gi...@apache.org>.
ashb commented on issue #10629:
URL: https://github.com/apache/airflow/issues/10629#issuecomment-733907248


   I think there may be one left -- in DagBag.collect_dags
   
   ```
           durations = (end_dttm - start_dttm).total_seconds()
           Stats.gauge('collect_dags', durations, 1)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org