You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2018/08/30 15:28:15 UTC
[GitHub] ashb commented on issue #3823: [AIRFLOW-2985] An operator for S3
object copying
ashb commented on issue #3823: [AIRFLOW-2985] An operator for S3 object copying
URL: https://github.com/apache/incubator-airflow/pull/3823#issuecomment-417361162
This would only work if the same connection ID has access to both buckets - and will make AWS do the copy (so we don't have to download and re-upload the file)
To support copying across roles/connections we could use something like https://pypi.org/project/smartopen/ to stream down and up (I have some code that does this if we want, but I think that is a different PR)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services