You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Ash Berlin-Taylor <as...@firemirror.com> on 2018/01/13 18:49:09 UTC
Re: Double Logging to GCS? (And to S3)
This is a known bug in 1.9.0 that we didn't manage to fix before release -- the fix for it was https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd <https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd> (merged in to master)
-ash
> On 12 Jan 2018, at 16:04, Kevin Lam <ke...@fathomhealth.co> wrote:
>
> Hi airflow folks,
>
> I'm currently using Airflow v1.9 and set up logging to GCS via
> http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
> into an issue where the entire local log appears to be being written to GCS
> are twice, any ideas what can cause this, or how to fix it?
>
> To give an example of what I'm observing, in the uploaded log file's you'll
> see the entire contents of what should be logged twice, eg. "Starting
> attempt X of Y" appears twice.
>
> Thanks in advance!
Re: Double Logging to GCS? (And to S3)
Posted by Ash Berlin-Taylor <as...@firemirror.com>.
Nope, probably not :(
-ash
> On 13 Jan 2018, at 18:51, Bolke de Bruin <bd...@gmail.com> wrote:
>
> We shouldn’t have missed that :-(
>
>> On 13 Jan 2018, at 19:49, Ash Berlin-Taylor <as...@firemirror.com> wrote:
>>
>> This is a known bug in 1.9.0 that we didn't manage to fix before release -- the fix for it was https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd <https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd> (merged in to master)
>>
>>
>> -ash
>>
>>> On 12 Jan 2018, at 16:04, Kevin Lam <ke...@fathomhealth.co> wrote:
>>>
>>> Hi airflow folks,
>>>
>>> I'm currently using Airflow v1.9 and set up logging to GCS via
>>> http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
>>> into an issue where the entire local log appears to be being written to GCS
>>> are twice, any ideas what can cause this, or how to fix it?
>>>
>>> To give an example of what I'm observing, in the uploaded log file's you'll
>>> see the entire contents of what should be logged twice, eg. "Starting
>>> attempt X of Y" appears twice.
>>>
>>> Thanks in advance!
>>
>
Re: Double Logging to GCS? (And to S3)
Posted by Bolke de Bruin <bd...@gmail.com>.
We shouldn’t have missed that :-(
> On 13 Jan 2018, at 19:49, Ash Berlin-Taylor <as...@firemirror.com> wrote:
>
> This is a known bug in 1.9.0 that we didn't manage to fix before release -- the fix for it was https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd <https://github.com/apache/incubator-airflow/commit/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd> (merged in to master)
>
>
> -ash
>
>> On 12 Jan 2018, at 16:04, Kevin Lam <ke...@fathomhealth.co> wrote:
>>
>> Hi airflow folks,
>>
>> I'm currently using Airflow v1.9 and set up logging to GCS via
>> http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
>> into an issue where the entire local log appears to be being written to GCS
>> are twice, any ideas what can cause this, or how to fix it?
>>
>> To give an example of what I'm observing, in the uploaded log file's you'll
>> see the entire contents of what should be logged twice, eg. "Starting
>> attempt X of Y" appears twice.
>>
>> Thanks in advance!
>