You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "EMT-Swaroop (via GitHub)" <gi...@apache.org> on 2024/05/09 12:13:47 UTC
[I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
EMT-Swaroop opened a new issue, #39522:
URL: https://github.com/apache/airflow/issues/39522
### Apache Airflow version
2.9.1
### If "Other Airflow 2 version" selected, which one?
2.9.0
### What happened?
According to airflow documentation, from 2.8.2 airflow version the metrics ti_failures, ti_successes, zombie_killed metrics need to have tagging task_id and dag_id
Airflow documentation:https://airflow.apache.org/docs/apache-airflow/2.9.1/administration-and-deployment/logging-monitoring/metrics.html
In our airflow environment we have upgraded airflow 2.9.1 and helm chart 1.13.1. Still we are not able to see the tagging for setting up alerts and monitoring at task level.
### What you think should happen instead?
The tagging of task-id and dag_id for metrics like ti_failures, ti_successes, zombie_killed need to be available so that we can locate the tasks accordingly to debug and we can have the alerting according to that.
### How to reproduce
If we use 2.8.1 or later version tagging need to be there for mentioned metrics. But those are not coming, only the counter values are getting produced by the airflow statsd.
### Operating System
Debian Bullseye
### Versions of Apache Airflow Providers
recently we have upgraded to 2.9.1.
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "EMT-Swaroop (via GitHub)" <gi...@apache.org>.
EMT-Swaroop commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2117385533
Hey @RNHTTR, Below are the list of few metrics that are missing tagging which were mentioned in the airflow documentation.
ti_failures
ti_successes
zombie_killed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "dirrao (via GitHub)" <gi...@apache.org>.
dirrao commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2118675198
@RNHTTR , @EMT-Swaroop
I will look into this radar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "EMT-Swaroop (via GitHub)" <gi...@apache.org>.
EMT-Swaroop commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2133292185
@dirrao , kindly help with the issue as are facing for monitoring the airlfow task fails due to the absence of task_ids
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "EMT-Swaroop (via GitHub)" <gi...@apache.org>.
EMT-Swaroop commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2120313306
@dirrao , we didn't customize any configuration in the statsd deployment, we are using the stasd exporter which coming from the parent airflow chart (1.13.1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "dirrao (via GitHub)" <gi...@apache.org>.
dirrao commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2118801521
If you are using statsd with influx backend store, you need to enable the `influxdb_tags_enabled` flag to propagate the tags. If not, Can you share the details about the metrics exporter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "EMT-Swaroop (via GitHub)" <gi...@apache.org>.
EMT-Swaroop commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2125950023
@dirrao any update?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2161828264
This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "RNHTTR (via GitHub)" <gi...@apache.org>.
RNHTTR commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2102933102
@dirrao can you take a look?
Also, it would probably be a good idea to add to the documentation what metrics tagging even is
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "lane-100 (via GitHub)" <gi...@apache.org>.
lane-100 commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2243957511
@EMT-Swaroop FWIW, it looks like the problem is caused by what's described in this issue, which hasn't (yet) been automatically closed: https://github.com/apache/airflow/issues/40027
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "lucasfcnunes (via GitHub)" <gi...@apache.org>.
lucasfcnunes commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2276928634
I'm looking forward to see `ti_failures` and `ti_successes` with the appropriate `dag_id` and `task_id` labels.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #39522:
URL: https://github.com/apache/airflow/issues/39522#issuecomment-2102548022
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org