You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "RNHTTR (via GitHub)" <gi...@apache.org> on 2023/04/18 17:47:53 UTC
[GitHub] [airflow] RNHTTR opened a new pull request, #30717: Update faq.rst about missing task logs
RNHTTR opened a new pull request, #30717:
URL: https://github.com/apache/airflow/pull/30717
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #30717: Update faq.rst about missing task logs
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #30717:
URL: https://github.com/apache/airflow/pull/30717#discussion_r1180275609
##########
docs/apache-airflow/faq.rst:
##########
@@ -441,6 +441,17 @@ If the tasks are not related by dependency, you will need to :ref:`build a custo
Airflow UI
^^^^^^^^^^
+Why did my task fail with no logs in the UI?
+--------------------------------------------
+
+Logs are [typically served when a task reaches a terminal state](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/logging-tasks.html#serving-logs-from-workers). Sometimes, a task's normal lifecycle is disrupted, and the task's
+worker is unable to write the task's logs. This typically happens for one of two reasons:
+
+1. [Zombie Tasks](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html#zombie-undead-tasks)
+2. Tasks failed after getting stuck in queued (Airflow 2.6.0+)
Review Comment:
What the Airflow 2.6.0+ means?
if task is stuck in queue it means it didn't reach worker. "extracting" it from the queue can be done with any Airflow version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30717: Update faq.rst about missing task logs
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30717:
URL: https://github.com/apache/airflow/pull/30717#issuecomment-1513586264
I like the description and good idea to mention it, maybe others would like to add something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RNHTTR commented on a diff in pull request #30717: Update faq.rst about missing task logs
Posted by "RNHTTR (via GitHub)" <gi...@apache.org>.
RNHTTR commented on code in PR #30717:
URL: https://github.com/apache/airflow/pull/30717#discussion_r1180454837
##########
docs/apache-airflow/faq.rst:
##########
@@ -441,6 +441,17 @@ If the tasks are not related by dependency, you will need to :ref:`build a custo
Airflow UI
^^^^^^^^^^
+Why did my task fail with no logs in the UI?
+--------------------------------------------
+
+Logs are [typically served when a task reaches a terminal state](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/logging-tasks.html#serving-logs-from-workers). Sometimes, a task's normal lifecycle is disrupted, and the task's
+worker is unable to write the task's logs. This typically happens for one of two reasons:
+
+1. [Zombie Tasks](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html#zombie-undead-tasks)
+2. Tasks failed after getting stuck in queued (Airflow 2.6.0+)
Review Comment:
#30375 will fail tasks that get stuck in queued for some reason. This will result in missing task logs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #30717: Update faq.rst about missing task logs
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #30717:
URL: https://github.com/apache/airflow/pull/30717#discussion_r1180457724
##########
docs/apache-airflow/faq.rst:
##########
@@ -441,6 +441,17 @@ If the tasks are not related by dependency, you will need to :ref:`build a custo
Airflow UI
^^^^^^^^^^
+Why did my task fail with no logs in the UI?
+--------------------------------------------
+
+Logs are [typically served when a task reaches a terminal state](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/logging-tasks.html#serving-logs-from-workers). Sometimes, a task's normal lifecycle is disrupted, and the task's
+worker is unable to write the task's logs. This typically happens for one of two reasons:
+
+1. [Zombie Tasks](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html#zombie-undead-tasks)
+2. Tasks failed after getting stuck in queued (Airflow 2.6.0+)
Review Comment:
So we should refer to the relevant Airflow setting task_queued_timeout and ask user to check if the threshold chosen is OK for them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #30717: Update faq.rst about missing task logs
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #30717:
URL: https://github.com/apache/airflow/pull/30717
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #30717: Update faq.rst about missing task logs
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on code in PR #30717:
URL: https://github.com/apache/airflow/pull/30717#discussion_r1171904431
##########
docs/apache-airflow/faq.rst:
##########
@@ -441,6 +441,17 @@ If the tasks are not related by dependency, you will need to :ref:`build a custo
Airflow UI
^^^^^^^^^^
+Why did my task fail with no logs in the UI?
+--------------------------------------------
Review Comment:
```suggestion
--------------------------------------------
```
nit
##########
docs/apache-airflow/faq.rst:
##########
@@ -441,6 +441,17 @@ If the tasks are not related by dependency, you will need to :ref:`build a custo
Airflow UI
^^^^^^^^^^
+Why did my task fail with no logs in the UI?
+--------------------------------------------
+Logs are [typically served when a task reaches a terminal state](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/logging-tasks.html#serving-logs-from-workers). Sometimes, a task's normal lifecycle is disrupted, and the task's
Review Comment:
Can you switch this link to be a ref, vs an external link?
##########
docs/apache-airflow/faq.rst:
##########
@@ -441,6 +441,17 @@ If the tasks are not related by dependency, you will need to :ref:`build a custo
Airflow UI
^^^^^^^^^^
+Why did my task fail with no logs in the UI?
+--------------------------------------------
+Logs are [typically served when a task reaches a terminal state](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/logging-tasks.html#serving-logs-from-workers). Sometimes, a task's normal lifecycle is disrupted, and the task's
+worker is unable to write the task's logs. This typically happens for one of two reasons:
+
+1. [Zombie Tasks](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html#zombie-undead-tasks)
+2. Tasks failed after getting stuck in queued (Airflow 2.6.0+)
+
+Setting retries for each task drastically reduces the chance that either of these problems impact a workflow.
+
Review Comment:
```suggestion
```
nit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org