You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Chaim (JIRA)" <ji...@apache.org> on 2019/01/17 07:55:00 UTC
[jira] [Created] (AIRFLOW-3720) GoogleCloudStorageToS3Operator -
incorrect folder compare
Chaim created AIRFLOW-3720:
------------------------------
Summary: GoogleCloudStorageToS3Operator - incorrect folder compare
Key: AIRFLOW-3720
URL: https://issues.apache.org/jira/browse/AIRFLOW-3720
Project: Apache Airflow
Issue Type: Bug
Components: boto3
Affects Versions: 1.10.0
Reporter: Chaim
Assignee: Chaim
Fix For: 1.10.2
the code that compares folders from gcp to s3 is incorrect.
the code is:
files = set(files) - set(existing_files)
but the list from gcp has a "/" to the name, for example: "myfolder/", while in s3 it does not have "/" so the folder is "myfolder"
the result is that the code tries to recopy the folder name but fails since it already exists
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)