You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/13 19:29:25 UTC

[GitHub] [airflow] AlexandreYang opened a new issue #11514: Missing Task Instance StatsD metrics for KubernetesExecutor

AlexandreYang opened a new issue #11514:
URL: https://github.com/apache/airflow/issues/11514


   <!--
   
   Welcome to Apache Airflow!  For a smooth issue process, try to answer the following questions.
   Don't worry if they're not all applicable; just try to include what you can :-)
   
   If you need to include code snippets or logs, please put them in fenced code
   blocks.  If they're super-long, please use the details tag like
   <details><summary>super-long log</summary> lots of stuff </details>
   
   Please delete these comment blocks before submitting the issue.
   
   -->
   
   **Description**
   
   Some StatsD metrics like sent from `taskinstance.py`:
   
   - [airflow.ti_failures](https://github.com/apache/airflow/blob/21066c2e6fecc52d5374209c2c4b64e3373a3e97/airflow/models/taskinstance.py#L1155)
   - [airflow.ti_successes](https://github.com/apache/airflow/blob/21066c2e6fecc52d5374209c2c4b64e3373a3e97/airflow/models/taskinstance.py#L1002)
   - [airflow.operator_failures_*](https://github.com/apache/airflow/blob/21066c2e6fecc52d5374209c2c4b64e3373a3e97/airflow/models/taskinstance.py#L1154)
   - [airflow.operator_successes_*](https://github.com/apache/airflow/blob/21066c2e6fecc52d5374209c2c4b64e3373a3e97/airflow/models/taskinstance.py#L1000-L1001)
   
   are *not* reported when using `KubernetesExecutor`.
   
   By digging into the code, it seems that methods like `TaskInstance._run_raw_task()` and `TaskInstance.handle_failure()` are not reached when using `KubernetesExecutor`.
   
   Is there a way to make those metrics also available for `KubernetesExecutor` ?
   
   <!-- A short description of your feature -->
   
   **Use case / motivation**
   
   Better observability. Those are useful metrics.
   
   <!-- What do you want to happen?
   
   Rather than telling us how you might implement this solution, try to take a
   step back and describe what you are trying to achieve.
   
   -->
   
   **Related Issues**
   
   It seems this also apply to CeleryExecutor and possibly other executors too.
   
   `CeleryExecutor` does report `airflow.ti_failures`, but not `airflow.ti_successes`.
   
   
   <!-- Is there currently another issue associated with this? -->
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #11514: Missing Task Instance StatsD metrics for KubernetesExecutor

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #11514:
URL: https://github.com/apache/airflow/issues/11514#issuecomment-707959997


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org