You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/30 13:27:53 UTC
[GitHub] [airflow] collinmcnulty opened a new issue, #26067: Include external_executor_id in zombie detection method
collinmcnulty opened a new issue, #26067:
URL: https://github.com/apache/airflow/issues/26067
### Description
Adjust the SimpleTaskInstance to include the external_executor_id so that it shows up when the zombie detection method prints the SimpleTaskInstance to logs.
### Use case/motivation
Since the zombie detection message originates in the dag file processor, further troubleshooting of the zombie task requires figuring out which worker was actually responsible for the task. Printing the external_executor_id makes it easier to find the task in a log aggregator like Kibana or Splunk than it is when using the combination of dag_id, task_id, logical_date, and map_index, at least for executors like Celery.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] collinmcnulty commented on issue #26067: Include external_executor_id in zombie detection method
Posted by GitBox <gi...@apache.org>.
collinmcnulty commented on issue #26067:
URL: https://github.com/apache/airflow/issues/26067#issuecomment-1231671604
I don't know if there are any secondary consequences to change SimpleTaskInstance, so I want to check with the community before I just add a new parameter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] wolfier commented on issue #26067: Include external_executor_id in zombie detection method
Posted by GitBox <gi...@apache.org>.
wolfier commented on issue #26067:
URL: https://github.com/apache/airflow/issues/26067#issuecomment-1233568516
The zombies query is a list of TaskInstances which has the attribute `external_executor_id`.
Without too much thought, you can probably add a new log line that prints the `external_executor_id` value.
```
self.log.error("Killing zombie job with external_executor_id: %s", ti.external_executor_id)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #26067: Include external_executor_id in zombie detection method
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #26067: Include external_executor_id in zombie detection method
URL: https://github.com/apache/airflow/issues/26067
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org