You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/29 15:14:45 UTC

[GitHub] [airflow] ashb opened a new pull request, #26779: Ensure the log messages from operators during parsing go somewhere

ashb opened a new pull request, #26779:
URL: https://github.com/apache/airflow/pull/26779

   While investigating #26599 and the change from AIP-45, I noticed that these warning messages weren't new! The only thing that was new was that we started seeing them.
   
   This is because the logger for BaseOperator and all subclasses is `airflow.task.operators`, and the `airflow.task` logger is not configured (with `set_context()`) until we have a TaskInstance, so it just dropped all messages on the floor!
   
   This changes it so that log messages are propagated to parent loggers by default, but when we configure a context (and thus have a file to write to) we stop that. A similar change was made for the `airflow.processor` (but that is unlikely to suffer the same fate)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ashb commented on a diff in pull request #26779: Ensure the log messages from operators during parsing go somewhere

Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26779:
URL: https://github.com/apache/airflow/pull/26779#discussion_r983770574


##########
airflow/utils/log/logging_mixin.py:
##########
@@ -179,15 +182,15 @@ def set_context(logger, value):
     :param logger: logger
     :param value: value to set
     """
-    _logger = logger
-    while _logger:
-        for handler in _logger.handlers:
+    while logger:
+        for handler in logger.handlers:
             # Not all handlers need to have context passed in so we ignore
             # the error when handlers do not have set_context defined.
             set_context = getattr(handler, 'set_context', None)
             if set_context:
-                set_context(value)
-        if _logger.propagate is True:
-            _logger = _logger.parent
+                if set_context(value) is DISABLE_PROPOGATE:

Review Comment:
   Can do, sure 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ashb commented on pull request #26779: Ensure the log messages from operators during parsing go somewhere

Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26779:
URL: https://github.com/apache/airflow/pull/26779#issuecomment-1262449537

   We shouldn't merge this to a release until we also fix the source of the issue (which is a bug, not user error)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] mertozb commented on a diff in pull request #26779: Ensure the log messages from operators during parsing go somewhere

Posted by GitBox <gi...@apache.org>.
mertozb commented on code in PR #26779:
URL: https://github.com/apache/airflow/pull/26779#discussion_r983819474


##########
airflow/utils/log/logging_mixin.py:
##########
@@ -179,15 +182,15 @@ def set_context(logger, value):
     :param logger: logger
     :param value: value to set
     """
-    _logger = logger
-    while _logger:
-        for handler in _logger.handlers:
+    while logger:
+        for handler in logger.handlers:
             # Not all handlers need to have context passed in so we ignore
             # the error when handlers do not have set_context defined.
             set_context = getattr(handler, 'set_context', None)
             if set_context:
-                set_context(value)
-        if _logger.propagate is True:
-            _logger = _logger.parent
+                if set_context(value) is DISABLE_PROPOGATE:

Review Comment:
   Generally the visibility of warnings may be added into cfg as a section like deprecation or dependency warnings so that we can able to path around them.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] kaxil commented on a diff in pull request #26779: Ensure the log messages from operators during parsing go somewhere

Posted by GitBox <gi...@apache.org>.
kaxil commented on code in PR #26779:
URL: https://github.com/apache/airflow/pull/26779#discussion_r983742204


##########
airflow/utils/log/logging_mixin.py:
##########
@@ -179,15 +182,15 @@ def set_context(logger, value):
     :param logger: logger
     :param value: value to set
     """
-    _logger = logger
-    while _logger:
-        for handler in _logger.handlers:
+    while logger:
+        for handler in logger.handlers:
             # Not all handlers need to have context passed in so we ignore
             # the error when handlers do not have set_context defined.
             set_context = getattr(handler, 'set_context', None)
             if set_context:
-                set_context(value)
-        if _logger.propagate is True:
-            _logger = _logger.parent
+                if set_context(value) is DISABLE_PROPOGATE:

Review Comment:
   Should we just chain this "if" with the "if" above?
   
   ```python
   if set_context and set_context(value) is DISABLE_PROPOGATE:
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ashb merged pull request #26779: Ensure the log messages from operators during parsing go somewhere

Posted by GitBox <gi...@apache.org>.
ashb merged PR #26779:
URL: https://github.com/apache/airflow/pull/26779


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ashb commented on a diff in pull request #26779: Ensure the log messages from operators during parsing go somewhere

Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26779:
URL: https://github.com/apache/airflow/pull/26779#discussion_r983896713


##########
airflow/utils/log/logging_mixin.py:
##########
@@ -179,15 +182,15 @@ def set_context(logger, value):
     :param logger: logger
     :param value: value to set
     """
-    _logger = logger
-    while _logger:
-        for handler in _logger.handlers:
+    while logger:
+        for handler in logger.handlers:
             # Not all handlers need to have context passed in so we ignore
             # the error when handlers do not have set_context defined.
             set_context = getattr(handler, 'set_context', None)
             if set_context:

Review Comment:
   ```suggestion
               if set_context and set_context(value) is DISABLE_PROPOGATE:
   ```
   



##########
airflow/utils/log/logging_mixin.py:
##########
@@ -179,15 +182,15 @@ def set_context(logger, value):
     :param logger: logger
     :param value: value to set
     """
-    _logger = logger
-    while _logger:
-        for handler in _logger.handlers:
+    while logger:
+        for handler in logger.handlers:
             # Not all handlers need to have context passed in so we ignore
             # the error when handlers do not have set_context defined.
             set_context = getattr(handler, 'set_context', None)
             if set_context:
-                set_context(value)
-        if _logger.propagate is True:
-            _logger = _logger.parent
+                if set_context(value) is DISABLE_PROPOGATE:
+                    logger.propagate = False

Review Comment:
   ```suggestion
                   logger.propagate = False
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org