You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/11/07 22:33:27 UTC
[GitHub] [airflow] blackmidnight opened a new issue, #17754: Support set logging level in Dag or Task layer
blackmidnight opened a new issue, #17754:
URL: https://github.com/apache/airflow/issues/17754
**Description**
Currently, airflow support to set logging level in airflow.cfg or by setting environment variable. But this setting will apply on all dags, and can not modify logging level for specific dag or task.
**Use case / motivation**
I want to ask if it's possible to provide parameter setting logging level in Dag or BaseOperator.
**Are you willing to submit a PR?**
Yes
**Related Issues**
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dgronskij commented on issue #17754: Support set logging level in Dag or Task layer
Posted by GitBox <gi...@apache.org>.
dgronskij commented on issue #17754:
URL: https://github.com/apache/airflow/issues/17754#issuecomment-1299881682
Yeah, sorry for the poorly worded question.
Here is our disposition:
- Airflow instance with no access to AF workers for developers
- 3rd party AF operators that do some logging
- Sometimes, there is a need to run specific DAG with custom log level in an ad-hoc manner, but we don't want to affect other runs since there are lot's of them and everybody would see verbose logs
Documentation that you cited suggests changing AF-wide configuration on the worker side. It seems that we could make a "debug node" worker, with debug level configured, and somehow route dags-to-be-debugged onto that.. but this is really complicated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17754: Support set logging level in Dag or Task layer
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17754:
URL: https://github.com/apache/airflow/issues/17754#issuecomment-1308816210
Ah yeah. That is actually good way of doing it. :). Good idea. Will close it but will keep the idea for similar cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dgronskij commented on issue #17754: Support set logging level in Dag or Task layer
Posted by GitBox <gi...@apache.org>.
dgronskij commented on issue #17754:
URL: https://github.com/apache/airflow/issues/17754#issuecomment-1308793699
@potiuk Thanks for the reponse!
I just came around the idea that what I wanted may be done via `on_execute_callback` / `pre_execute` callback functions. At the end of the day I'm not sure that there is a need of extra parameters since callbacks are the most flexible way to do anything.
Probably, making parameters for `logging.rootLogger` loglevel or `logging.basicConfig` kwargs would be a good compomise, but not sure -- my case is quite niche, as I see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17754: Support set logging level in Dag or Task layer
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17754:
URL: https://github.com/apache/airflow/issues/17754#issuecomment-1306320513
Ok. This sounds more interestin and cool. Marked is as good first issue, maybe someone would like to attempt implementing it? Note that this is an open-source. Opening feature proposal like this does not mean it will be implemented. Someone must pick an interest an implement it. If you or your team would like to develop or contribute such fix, this is the fastest way - otehrwise it will have to wait for someone to pick it up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dgronskij commented on issue #17754: Support set logging level in Dag or Task layer
Posted by GitBox <gi...@apache.org>.
dgronskij commented on issue #17754:
URL: https://github.com/apache/airflow/issues/17754#issuecomment-1298264347
Hey, how is it supposed to be done on the client side without changing the operators' code (which are distributed as a package)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17754: Support set logging level in Dag or Task layer
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17754:
URL: https://github.com/apache/airflow/issues/17754#issuecomment-1298407356
Did you check https://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/logging-tasks.html#advanced-configuration ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17754: Support set logging level in Dag or Task layer
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17754: Support set logging level in Dag or Task layer
URL: https://github.com/apache/airflow/issues/17754
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org