You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2023/01/09 22:04:42 UTC
[GitHub] [airflow] snjypl opened a new pull request, #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
snjypl opened a new pull request, #28817:
URL: https://github.com/apache/airflow/pull/28817
for k8s executor while trying to view the task log from the UI. we are getting the following error.
[ note: i have set `delete_worker_pods` to `False` so that the pod is not deleted after completion]
```
airflow-webserver-5bf48475c-zdjxv
*** Trying to get logs (last 100 lines) from worker pod airflow-webserver-5bf48475c-zdjxv ***
*** Unable to fetch logs from worker pod airflow-webserver-5bf48475c-zdjxv ***
('Cannot find pod for ti %s', <TaskInstance: dataset_produces_2.producing_task_2 manual__2022-12-28T21:13:27.229615+00:00 [success]>)
```
i think, the issue is, while calling `PodGenerator.build_selector_for_k8s_executor_pod` we are passing `ti.try_number` instead of the `try_number` that was passed to the `_read` method.
https://github.com/apache/airflow/blob/3ececb2c79307bd943aad116d7b0b5933af8185a/airflow/utils/log/file_task_handler.py#L167
https://github.com/apache/airflow/blob/3ececb2c79307bd943aad116d7b0b5933af8185a/airflow/utils/log/file_task_handler.py#L215-L222
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on a diff in pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #28817:
URL: https://github.com/apache/airflow/pull/28817#discussion_r1106397651
##########
airflow/executors/base_executor.py:
##########
@@ -356,13 +356,13 @@ def execute_async(
"""
raise NotImplementedError()
- def get_task_log(self, ti: TaskInstance) -> tuple[list[str], list[str]]:
+ def get_task_log(self, ti: TaskInstance, try_number: int) -> tuple[list[str], list[str]]:
"""
Review Comment:
wait this bit doesn't make sense to me. if we already have the TI, why is it we also need to supply the try number?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal merged pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #28817:
URL: https://github.com/apache/airflow/pull/28817
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on a diff in pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on code in PR #28817:
URL: https://github.com/apache/airflow/pull/28817#discussion_r1106495625
##########
airflow/executors/base_executor.py:
##########
@@ -356,13 +356,13 @@ def execute_async(
"""
raise NotImplementedError()
- def get_task_log(self, ti: TaskInstance) -> tuple[list[str], list[str]]:
+ def get_task_log(self, ti: TaskInstance, try_number: int) -> tuple[list[str], list[str]]:
"""
Review Comment:
There is some explanation in the code for `read`: https://github.com/apache/airflow/blob/c266aacf39a2f01cb3f91740f448934c7b3fd7be/airflow/utils/log/file_task_handler.py#L376-L410
`try_number` is plumbed in from `read` to `_read` and then should have been sent to the executors, but it was a miss:
https://github.com/apache/airflow/blob/c266aacf39a2f01cb3f91740f448934c7b3fd7be/airflow/utils/log/file_task_handler.py#L274-L315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #28817:
URL: https://github.com/apache/airflow/pull/28817#issuecomment-1378616691
Paging @dstandish
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on PR #28817:
URL: https://github.com/apache/airflow/pull/28817#issuecomment-1426072031
Looks reasonable to me, but needs a rebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #28817:
URL: https://github.com/apache/airflow/pull/28817#issuecomment-1461070967
Shall we just merge this fix then @potiuk @eladkal? Worst case we can always follow-up with more changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #28817:
URL: https://github.com/apache/airflow/pull/28817#issuecomment-1449173248
Anyone have more feedback for this one or shall I merge it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #28817:
URL: https://github.com/apache/airflow/pull/28817#issuecomment-1449432297
@dstandish ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pierrejeambrun commented on pull request #28817: Fix Unable to fetch logs from worker pod error in UI for k8s executor
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #28817:
URL: https://github.com/apache/airflow/pull/28817#issuecomment-1481590778
Conflicting, requires https://github.com/apache/airflow/pull/29482 and https://github.com/apache/airflow/pull/28161. Marking for 2.6.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org