You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Kevin Lam <ke...@fathomhealth.co> on 2018/01/12 16:04:30 UTC
Double Logging to GCS?
Hi airflow folks,
I'm currently using Airflow v1.9 and set up logging to GCS via
http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
into an issue where the entire local log appears to be being written to GCS
are twice, any ideas what can cause this, or how to fix it?
To give an example of what I'm observing, in the uploaded log file's you'll
see the entire contents of what should be logged twice, eg. "Starting
attempt X of Y" appears twice.
Thanks in advance!
Re: Double Logging to GCS? (And to S3)
Posted by Ash Berlin-Taylor <as...@firemirror.com>.
Nope, probably not :(
-ash
> On 13 Jan 2018, at 18:51, Bolke de Bruin <bd...@gmail.com> wrote:
>
> We shouldn’t have missed that :-(
>
>> On 13 Jan 2018, at 19:49, Ash Berlin-Taylor <as...@firemirror.com> wrote:
>>
>> This is a known bug in 1.9.0 that we didn't manage to fix before release -- the fix for it was https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd <https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd> (merged in to master)
>>
>>
>> -ash
>>
>>> On 12 Jan 2018, at 16:04, Kevin Lam <ke...@fathomhealth.co> wrote:
>>>
>>> Hi airflow folks,
>>>
>>> I'm currently using Airflow v1.9 and set up logging to GCS via
>>> http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
>>> into an issue where the entire local log appears to be being written to GCS
>>> are twice, any ideas what can cause this, or how to fix it?
>>>
>>> To give an example of what I'm observing, in the uploaded log file's you'll
>>> see the entire contents of what should be logged twice, eg. "Starting
>>> attempt X of Y" appears twice.
>>>
>>> Thanks in advance!
>>
>
Re: Double Logging to GCS? (And to S3)
Posted by Bolke de Bruin <bd...@gmail.com>.
We shouldn’t have missed that :-(
> On 13 Jan 2018, at 19:49, Ash Berlin-Taylor <as...@firemirror.com> wrote:
>
> This is a known bug in 1.9.0 that we didn't manage to fix before release -- the fix for it was https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd <https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd> (merged in to master)
>
>
> -ash
>
>> On 12 Jan 2018, at 16:04, Kevin Lam <ke...@fathomhealth.co> wrote:
>>
>> Hi airflow folks,
>>
>> I'm currently using Airflow v1.9 and set up logging to GCS via
>> http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
>> into an issue where the entire local log appears to be being written to GCS
>> are twice, any ideas what can cause this, or how to fix it?
>>
>> To give an example of what I'm observing, in the uploaded log file's you'll
>> see the entire contents of what should be logged twice, eg. "Starting
>> attempt X of Y" appears twice.
>>
>> Thanks in advance!
>
Re: Double Logging to GCS? (And to S3)
Posted by Ash Berlin-Taylor <as...@firemirror.com>.
This is a known bug in 1.9.0 that we didn't manage to fix before release -- the fix for it was https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd <https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd> (merged in to master)
-ash
> On 12 Jan 2018, at 16:04, Kevin Lam <ke...@fathomhealth.co> wrote:
>
> Hi airflow folks,
>
> I'm currently using Airflow v1.9 and set up logging to GCS via
> http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
> into an issue where the entire local log appears to be being written to GCS
> are twice, any ideas what can cause this, or how to fix it?
>
> To give an example of what I'm observing, in the uploaded log file's you'll
> see the entire contents of what should be logged twice, eg. "Starting
> attempt X of Y" appears twice.
>
> Thanks in advance!