You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "mattp- (via GitHub)" <gi...@apache.org> on 2025/10/01 20:24:25 UTC
[PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
mattp- opened a new pull request, #56313:
URL: https://github.com/apache/airflow/pull/56313
if not set, a stacktrace like the following can occur when the no-exception path is executed:
```
2025-10-01T20:14:42.384531Z [info ] Remote task logs will not be available due to an error: module 'mse.airflow.log_config' has no attribute 'REMOTE_TASK_LOG' [airflow.logging_config] loc=logging_config.py:82
...
if (remote := load_remote_log_handler()) and (remote_processors := getattr(remote, "processors")):
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/bb/bin/package/m/mse-airflow-base/0.0.25+devise/libexec/mse-airflow-base/python/lib/python3.12/site-packages/airflow/sdk/log.py", line 171, in load_remote_log_handler
return airflow.logging_config.REMOTE_TASK_LOG
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/bb/bin/package/m/mse-airflow-base/0.0.25+devise/libexec/mse-airflow-base/python/lib/python3.12/site-packages/airflow/logging_config.py", line 43, in __getattr__
return REMOTE_TASK_LOG
^^^^^^^^^^^^^^^
NameError: name 'REMOTE_TASK_LOG' is not defined
```
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on PR #56313:
URL: https://github.com/apache/airflow/pull/56313#issuecomment-3372808751
I'm not sure at all yet. Though I think it very likely that that file will not be used much longer anyway (We are slowly working on converting everything in Airflow to use structlog, it started in 3.0, and a big chunk got done in 3.1 as well)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
Posted by "mattp- (via GitHub)" <gi...@apache.org>.
mattp- commented on PR #56313:
URL: https://github.com/apache/airflow/pull/56313#issuecomment-3372669007
ah i see. that's fine too; though it feels a bit action at a distance to specify a class in the config, except you actually need a glob import for things to work properly. can we maybe provide this behavior by subclassing instead?
if we want to make the * import the standard i think there are some doc examples in airflow 2 style, I can send a PR to update
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on PR #56313:
URL: https://github.com/apache/airflow/pull/56313#issuecomment-3372595908
I'm tempted to say That is a bug in your logging config (and you will likely run in to other problems too. and a difference/incompatibility between Airflow 2 and Airflow 3.
```
from airflow.config_templates.airflow_local_settings import *
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #56313: airflow.logging_config: set default for REMOTE_TASK_LOG
URL: https://github.com/apache/airflow/pull/56313
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on PR #56313:
URL: https://github.com/apache/airflow/pull/56313#issuecomment-3371911864
How do you actually hit this codepath though? Airflow clearly works without setting a remote logging provider...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #56313:
URL: https://github.com/apache/airflow/pull/56313#issuecomment-3619020727
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow.logging_config: set default for REMOTE_TASK_LOG [airflow]
Posted by "mattp- (via GitHub)" <gi...@apache.org>.
mattp- commented on PR #56313:
URL: https://github.com/apache/airflow/pull/56313#issuecomment-3372371447
@ashb It seems tied to using a custom logging class to reproduce:
logging_config_class = bloomberg.mse.airflow.log_config.LOGGING_CONFIG , where the contents of that is something like (truncated)
```
❯ cat src/bloomberg/mse/airflow/log_config.py
import os
from copy import deepcopy
from airflow.config_templates.airflow_local_settings import DEFAULT_LOGGING_CONFIG
# Using deepcopy will allow us to keep up with the new features that airflow exposes while
# keeping our customized logs
# https://github.com/apache/airflow/blob/v2-5-stable/airflow/config_templates/airflow_local_settings.py#L66
# Lists all default logging config airflow is using, which already taking account of env var
LOGGING_CONFIG = deepcopy(DEFAULT_LOGGING_CONFIG)
```
it doesn't happen if logging_config_class is unset
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org