You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by "Miller, Robin" <Ro...@affiliate.oliverwyman.com> on 2017/02/01 10:56:07 UTC
Airflow Logging Updates
Hi All,
A while ago we took over this issue: https://issues.apache.org/jira/browse/AIRFLOW-409, which revolved around avoiding making changes to the root python logging configuration so as to avoid any accidental side effects that this could produce in other python applications.
The approach we decided on was to avoid use of the root logger entirely and produce a single configuration point for the logging to allow it to be configured pretty much however is desired in one place (with currently fairly simple options, but this could be made as complex or configurable as desired with changes to only a single file).
As you might expect this affects a lot of the code, as it requires changing every log statement that's contains "logging.Info", "logging.Error", etc.
The pull request for this, https://github.com/apache/incubator-airflow/pull/1921, has been open for almost 2 months now and unsurprisingly now has merge conflicts. We'll happily clean these up, but would prefer to do so at the time this is being reviewed, otherwise more will inevitably appear.
So I'm wondering, is this issue/pull request likely to get any attention soon? Or is this change simply unwanted?
Thanks,
Robin Miller
OLIVER WYMAN
robin.miller@affiliate.oliverwyman.com<ma...@affiliate.oliverwyman.com>
www.oliverwyman.com<http://www.oliverwyman.com/>
________________________________
This e-mail and any attachments may be confidential or legally privileged. If you received this message in error or are not the intended recipient, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained herein. Please inform us of the erroneous delivery by return e-mail. Thank you for your cooperation.
Re: Airflow Logging Updates
Posted by "Miller, Robin" <Ro...@affiliate.oliverwyman.com>.
Hi Alex,
Thanks for your interest in this.
Hopefully this will be a good first step for any further logging changes in the future, so will be helpful with logging to Google Cloud.
To be clear, I wasn't expecting this go into the 1.8 release at this stage. Having it in for the next release after that would be nice, though.
Thanks,
Robin Miller
OLIVER WYMAN
robin.miller@affiliate.oliverwyman.com<ma...@affiliate.oliverwyman.com>
www.oliverwyman.com<http://www.oliverwyman.com/>
________________________________
From: Alex Van Boxel <al...@vanboxel.be>
Sent: 01 February 2017 12:55:42
To: dev@airflow.incubator.apache.org
Subject: Re: Airflow Logging Updates
Hey Robin,
I also have an interest in logging, but to get Google Cloud logging to
work. I'm happy to work with you on this. But *first* the 1.8 needs to go
out.
But I don't think this will go on the patch branch, but maybe for the next
release. I don't know what the other people think.
On Wed, Feb 1, 2017 at 11:58 AM Miller, Robin <
Robin.Miller@affiliate.oliverwyman.com> wrote:
> Hi All,
>
>
> A while ago we took over this issue:
> https://issues.apache.org/jira/browse/AIRFLOW-409, which revolved around
> avoiding making changes to the root python logging configuration so as to
> avoid any accidental side effects that this could produce in other python
> applications.
>
>
> The approach we decided on was to avoid use of the root logger entirely
> and produce a single configuration point for the logging to allow it to be
> configured pretty much however is desired in one place (with currently
> fairly simple options, but this could be made as complex or configurable as
> desired with changes to only a single file).
>
>
> As you might expect this affects a lot of the code, as it requires
> changing every log statement that's contains "logging.Info",
> "logging.Error", etc.
>
>
> The pull request for this,
> https://github.com/apache/incubator-airflow/pull/1921, has been open for
> almost 2 months now and unsurprisingly now has merge conflicts. We'll
> happily clean these up, but would prefer to do so at the time this is being
> reviewed, otherwise more will inevitably appear.
>
>
> So I'm wondering, is this issue/pull request likely to get any attention
> soon? Or is this change simply unwanted?
>
>
> Thanks,
>
> Robin Miller
> OLIVER WYMAN
> robin.miller@affiliate.oliverwyman.com<mailto:
> robin.miller@affiliate.oliverwyman.com>
> www.oliverwyman.com<http://www.oliverwyman.com/>
>
>
> ________________________________
> This e-mail and any attachments may be confidential or legally privileged.
> If you received this message in error or are not the intended recipient,
> you should destroy the e-mail message and any attachments or copies, and
> you are prohibited from retaining, distributing, disclosing or using any
> information contained herein. Please inform us of the erroneous delivery by
> return e-mail. Thank you for your cooperation.
>
--
_/
_/ Alex Van Boxel
________________________________
This e-mail and any attachments may be confidential or legally privileged. If you received this message in error or are not the intended recipient, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained herein. Please inform us of the erroneous delivery by return e-mail. Thank you for your cooperation.
Re: Airflow Logging Updates
Posted by Alex Van Boxel <al...@vanboxel.be>.
Hey Robin,
I also have an interest in logging, but to get Google Cloud logging to
work. I'm happy to work with you on this. But *first* the 1.8 needs to go
out.
But I don't think this will go on the patch branch, but maybe for the next
release. I don't know what the other people think.
On Wed, Feb 1, 2017 at 11:58 AM Miller, Robin <
Robin.Miller@affiliate.oliverwyman.com> wrote:
> Hi All,
>
>
> A while ago we took over this issue:
> https://issues.apache.org/jira/browse/AIRFLOW-409, which revolved around
> avoiding making changes to the root python logging configuration so as to
> avoid any accidental side effects that this could produce in other python
> applications.
>
>
> The approach we decided on was to avoid use of the root logger entirely
> and produce a single configuration point for the logging to allow it to be
> configured pretty much however is desired in one place (with currently
> fairly simple options, but this could be made as complex or configurable as
> desired with changes to only a single file).
>
>
> As you might expect this affects a lot of the code, as it requires
> changing every log statement that's contains "logging.Info",
> "logging.Error", etc.
>
>
> The pull request for this,
> https://github.com/apache/incubator-airflow/pull/1921, has been open for
> almost 2 months now and unsurprisingly now has merge conflicts. We'll
> happily clean these up, but would prefer to do so at the time this is being
> reviewed, otherwise more will inevitably appear.
>
>
> So I'm wondering, is this issue/pull request likely to get any attention
> soon? Or is this change simply unwanted?
>
>
> Thanks,
>
> Robin Miller
> OLIVER WYMAN
> robin.miller@affiliate.oliverwyman.com<mailto:
> robin.miller@affiliate.oliverwyman.com>
> www.oliverwyman.com<http://www.oliverwyman.com/>
>
>
> ________________________________
> This e-mail and any attachments may be confidential or legally privileged.
> If you received this message in error or are not the intended recipient,
> you should destroy the e-mail message and any attachments or copies, and
> you are prohibited from retaining, distributing, disclosing or using any
> information contained herein. Please inform us of the erroneous delivery by
> return e-mail. Thank you for your cooperation.
>
--
_/
_/ Alex Van Boxel