You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Jarek Potiuk (JIRA)" <ji...@apache.org> on 2019/04/27 18:39:00 UTC

[jira] [Resolved] (AIRFLOW-3720) GoogleCloudStorageToS3Operator - incorrect folder compare

     [ https://issues.apache.org/jira/browse/AIRFLOW-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarek Potiuk resolved AIRFLOW-3720.
-----------------------------------
    Resolution: Fixed

> GoogleCloudStorageToS3Operator -  incorrect folder compare
> ----------------------------------------------------------
>
>                 Key: AIRFLOW-3720
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3720
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: aws
>    Affects Versions: 1.10.0
>            Reporter: Chaim
>            Assignee: Chaim
>            Priority: Major
>             Fix For: 2.0.0
>
>
> the code that compares folders from gcp to s3 is incorrect.
> the code is:
> files = set(files) - set(existing_files)
> but the list from gcp  has a "/" to the name, for example: "myfolder/", while in s3 it does not have "/" so the folder is "myfolder"
> the result is that the code tries to recopy the folder name but fails since it already exists



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)