[GitHub] [airflow] uranusjr commented on a diff in pull request #28440: Propagate logs to stdout when in k8s executor pod

[GitBox <gi...@apache.org>]

uranusjr commented on code in PR #28440:
URL: https://github.com/apache/airflow/pull/28440#discussion_r1060270338


##########
airflow/cli/commands/task_command.py:
##########
@@ -281,38 +283,62 @@ def _extract_external_executor_id(args) -> str | None:
 
 
 @contextmanager
-def _capture_task_logs(ti: TaskInstance) -> Generator[None, None, None]:
+def _move_task_handlers_to_root(ti: TaskInstance) -> Generator[None, None, None]:
     """
-    Manage logging context for a task run.
+    Move handlers for task logging to root logger.
 
-    - Replace the root logger configuration with the airflow.task configuration
-      so we can capture logs from any custom loggers used in the task.
+    We want anything logged during task run to be propagated to task log handlers.
+    If running in a k8s executor pod, also keep the stream handler on root logger
+    so that logs are still emitted to stdout.
+    """
+    # nothing to do
+    if not ti.log.handlers or settings.DONOT_MODIFY_HANDLERS:
+        yield
+        return
+
+    def get_console_handler(logger):
+        for h in logger.handlers:
+            if h.name == "console":
+                return h
+
+    def ensure_handler(logger, handler):
+        if not handler:
+            return
+        if handler not in logger.handlers:
+            logger.addHandler(handler)
+
+    # Move task handlers to root and reset task logger and restore original logger settings after exit.
+    # If k8s executor, we need to ensure that root logger has a console handler, so that
+    # task logs propagate to stdout (this is how webserver retrieves them while task is running).
+    root_logger = logging.getLogger()
+    console_handler = get_console_handler(root_logger)

Review Comment:
   Can be simplified into
   
   ```python
   console_handler = next((h for h in logger.handlers if h.name == "console"), None)
   ```
   
   Also it seems logic in `ensure_handler` can also be inlined?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org