You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "lanxinxs (via GitHub)" <gi...@apache.org> on 2024/12/05 08:51:54 UTC
[I] Unable to view trigger log in real-time in Airflow UI when task is deferred [airflow]
lanxinxs opened a new issue, #44683:
URL: https://github.com/apache/airflow/issues/44683
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.6.3
### What happened?
when task is deferred, the webserver UI is unable to read trigger log:
`*** Could not read served logs: Client error '404 NOT FOUND' for url 'http://<triggerer hostname>:8794/log/dag_id=xxx/run_id=xxx/task_id=xxx/attempt=18.log.trigger.3509.log'
`
after task is completed, the above error message will disappear, it says the trigger log is found on s3 (we enable remote logging):
`*** Found logs in s3:
*** * s3://<bucket>/path_to_log/attempt=18.log
*** * s3://<bucket>/path_to_log/attempt=18.log.trigger.3509.log`
and the content of the trigger log is displayed with the task log in the UI
`[2024-12-05, 07:34:00 UTC] {triggerer_job_runner.py:616} INFO - Trigger xxx/-1/18 (ID 160) fired: TriggerEvent<{'state': 'SUCCESS', 'message': None, 'extra_data': {...}}>
`
While task is deferred I exec into the triggerer pod, I can't find the expected xxx/attempt=18.log.trigger.3509.log in log folder; while after the task completes, I can find this log in the folder
### What you think should happen instead?
while task is deferred, I expect to view the corresponding trigger log in real-time, like the video [here](https://github.com/apache/airflow/pull/27758)
### How to reproduce
run a DAG with deferrable task, and view the task log when it is deferred
### Operating System
Ubuntu
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.1.0
apache-airflow-providers-celery==3.2.0
apache-airflow-providers-cncf-kubernetes==7.0.0
apache-airflow-providers-common-sql==1.5.1
apache-airflow-providers-docker==3.7.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
triggerer pod has open 8794 port, and has a service listening on that port
`triggerer ClusterIP None <none> 8794/TCP 22d
`
and webserver secret key has configured properly.
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to view trigger log in real-time in Airflow UI when task is deferred [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #44683: Unable to view trigger log in real-time in Airflow UI when task is deferred
URL: https://github.com/apache/airflow/issues/44683
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to view trigger log in real-time in Airflow UI when task is deferred [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #44683:
URL: https://github.com/apache/airflow/issues/44683#issuecomment-2519654096
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unable to view trigger log in real-time in Airflow UI when task is deferred [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #44683:
URL: https://github.com/apache/airflow/issues/44683#issuecomment-2522844988
Likely duplicate of https://github.com/apache/airflow/pull/41272 -> please upgrade to latest 2.10 (2.10.3 now) to check if it is fixed there. Closing as duplicate - if it appears still after upgrade, we can always re-open it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org