You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ashb (via GitHub)" <gi...@apache.org> on 2025/03/13 15:00:38 UTC
[PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
ashb opened a new pull request, #47731:
URL: https://github.com/apache/airflow/pull/47731
This is needed so that when the KubeExecutor is asked to get logs for a
running pod they show up in the pod.
This changes the `airflow.sdk.execution_time.execute_workload` entrypoint to
a. produce all logs as JSON, and
b. ask the SDK to send all output log messages to the top level logger to, so
they appear on stdout.
In order to make the output a bit nicer this also tidies up/removes some of
the logging from dispose_orm so that it doesn't "pollute" the logs (this was
required as due to the current hack we have to upload remote logs, we ended up
alling dispose_orm at the end and that _wasn't_ JSON formatted).
Closes #46894
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham merged PR #47731:
URL: https://github.com/apache/airflow/pull/47731
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #47731:
URL: https://github.com/apache/airflow/pull/47731#discussion_r1993741232
##########
task-sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -371,6 +371,9 @@ class WatchedSubprocess:
log: FilteringBoundLogger
+ verbose_logs_to_main_logger: bool = False
+ """Duplicate log messages to main logger, or only send them to ``self.log``."""
Review Comment:
what is "main logger"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #47731:
URL: https://github.com/apache/airflow/pull/47731#discussion_r1993740005
##########
task-sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -371,6 +371,9 @@ class WatchedSubprocess:
log: FilteringBoundLogger
+ verbose_logs_to_main_logger: bool = False
Review Comment:
the verbosity word here is probably confusing. verbosity connotes log level. but this is just about echoing to stdout, or teeing to stdout, irrespective of log level
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #47731:
URL: https://github.com/apache/airflow/pull/47731#discussion_r1993746067
##########
task-sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -445,17 +448,22 @@ def _register_pipe_readers(self, stdout: socket, stderr: socket, requests: socke
# alternatives are used automatically) -- this is a way of having "event-based" code, but without
# needing full async, to read and process output from each socket as it is received.
- self.selector.register(stdout, selectors.EVENT_READ, self._create_socket_handler(self.log, "stdout"))
+ target_loggers: tuple[FilteringBoundLogger, ...] = (self.log,)
+ if self.verbose_logs_to_main_logger:
+ target_loggers += (log,)
Review Comment:
it seems odd that `self.log` would not go to stdout but `log` would go to stdout. so if i use the global `log`, it will not go to task logs?
shouldn't it be more idiot proof than that, in the sense that, whichever log you a re using, it would go to both?
if `log` is a special "stdout only" logger, it should probably be named specially to reflect that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #47731:
URL: https://github.com/apache/airflow/pull/47731#discussion_r1993751908
##########
task-sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -445,17 +448,22 @@ def _register_pipe_readers(self, stdout: socket, stderr: socket, requests: socke
# alternatives are used automatically) -- this is a way of having "event-based" code, but without
# needing full async, to read and process output from each socket as it is received.
- self.selector.register(stdout, selectors.EVENT_READ, self._create_socket_handler(self.log, "stdout"))
+ target_loggers: tuple[FilteringBoundLogger, ...] = (self.log,)
+ if self.verbose_logs_to_main_logger:
+ target_loggers += (log,)
Review Comment:
`self.log` is the process specific logger, which is configured to write to the file.
`log` is the top level module scoped generic logger.
Would renaming `self.log` to `self.process_log` or `process_logger` help do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #47731:
URL: https://github.com/apache/airflow/pull/47731#discussion_r1993742984
##########
task-sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -371,6 +371,9 @@ class WatchedSubprocess:
log: FilteringBoundLogger
+ verbose_logs_to_main_logger: bool = False
Review Comment:
mmm yes good shout. Is `subprocess_logs_to_stdout` better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Send task logs from KubeExecutor tasks to pod stdout too [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #47731:
URL: https://github.com/apache/airflow/pull/47731#discussion_r1993766548
##########
task-sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -445,17 +448,22 @@ def _register_pipe_readers(self, stdout: socket, stderr: socket, requests: socke
# alternatives are used automatically) -- this is a way of having "event-based" code, but without
# needing full async, to read and process output from each socket as it is received.
- self.selector.register(stdout, selectors.EVENT_READ, self._create_socket_handler(self.log, "stdout"))
+ target_loggers: tuple[FilteringBoundLogger, ...] = (self.log,)
+ if self.verbose_logs_to_main_logger:
+ target_loggers += (log,)
Review Comment:
Yeah `self.process_log` is probably better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org