You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jack (JIRA)" <ji...@apache.org> on 2019/01/17 13:08:00 UTC
[jira] [Issue Comment Deleted] (AIRFLOW-3720)
GoogleCloudStorageToS3Operator - incorrect folder compare
[ https://issues.apache.org/jira/browse/AIRFLOW-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jack updated AIRFLOW-3720:
--------------------------
Comment: was deleted
(was: The bug was fixed in 1.10.2 [https://github.com/apache/airflow/pull/4371] )
> GoogleCloudStorageToS3Operator - incorrect folder compare
> ----------------------------------------------------------
>
> Key: AIRFLOW-3720
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3720
> Project: Apache Airflow
> Issue Type: Bug
> Components: boto3
> Affects Versions: 1.10.0
> Reporter: Chaim
> Assignee: Chaim
> Priority: Major
> Fix For: 1.10.2
>
>
> the code that compares folders from gcp to s3 is incorrect.
> the code is:
> files = set(files) - set(existing_files)
> but the list from gcp has a "/" to the name, for example: "myfolder/", while in s3 it does not have "/" so the folder is "myfolder"
> the result is that the code tries to recopy the folder name but fails since it already exists
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)