You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "manipatnam (via GitHub)" <gi...@apache.org> on 2025/07/31 12:50:10 UTC
[I] Airflow UI does not show triggerer logs during runtime under Kubernetes Executor [airflow]
manipatnam opened a new issue, #53974:
URL: https://github.com/apache/airflow/issues/53974
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.10.1
### What happened?
I created a simple DAG using TimeDeltaSensorAsync to test async sensor behavior across different Airflow executors and Astro Runtime versions. The DAG uses mode="reschedule" and a delta of 10 minutes.
When running this DAG:
With the Celery Executor (Astro Runtime 9.19.0):
The triggerer logs for TimeDeltaSensorAsync are visible in the Airflow UI under task logs during runtime.
These logs also appear inside the triggerer container.
With the Kubernetes Executor (Astro Runtime 9.19.0 and 12.1.0):
The triggerer logs are not visible in the Airflow UI during runtime.
However, the logs are present inside the triggerer container when inspected directly (e.g., via kubectl logs).
This suggests that the triggerer logs are not being forwarded or uploaded properly to the remote logging backend (in our case, S3) when using the Kubernetes Executor.
### What you think should happen instead?
Runtime logs for tasks should be visible in the Airflow UI when using the Kubernetes Executor — just like they are with the Celery Executor. Logs from the initial task attempt should be uploaded to remote storage and available during runtime.
### How to reproduce
Create a DAG using TimeDeltaSensorAsync with mode="reschedule" and a short delta, e.g., 10 minutes:
```
from airflow import DAG
from airflow.sensors.time_delta import TimeDeltaSensorAsync
from datetime import datetime, timedelta
with DAG(
dag_id="test_timedelta_sensor_async",
start_date=datetime(2025, 7, 15),
schedule_interval=None,
catchup=False,
) as dag:
TimeDeltaSensorAsync(
task_id="wait_for_time",
delta=timedelta(minutes=10),
mode="reschedule",
)
```
Run this DAG using the Kubernetes Executor
Observe the task logs in the Airflow UI while the sensor is in deferred mode.
### Operating System
debian
### Versions of Apache Airflow Providers
_No response_
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow UI does not show triggerer logs during runtime under Kubernetes Executor [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #53974:
URL: https://github.com/apache/airflow/issues/53974#issuecomment-4402222690
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow UI does not show triggerer logs during runtime under Kubernetes Executor [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #53974:
URL: https://github.com/apache/airflow/issues/53974#issuecomment-3937754124
This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow UI does not show triggerer logs during runtime under Kubernetes Executor [airflow]
Posted by "wolfier (via GitHub)" <gi...@apache.org>.
wolfier commented on issue #53974:
URL: https://github.com/apache/airflow/issues/53974#issuecomment-3281362090
I believe the trigger logs are generally only uploaded when the trigger is fired. Additionally, Airflow does reach out to the triggerer when there is an active trigger to fetch the logs. The issue is due to some misconfigured networking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow UI does not show triggerer logs during runtime under Kubernetes Executor [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #53974: Airflow UI does not show triggerer logs during runtime under Kubernetes Executor
URL: https://github.com/apache/airflow/issues/53974
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org