You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/26 22:00:31 UTC
[GitHub] [airflow] YingyingPeng22 opened a new pull request #13919: #9803 fix bug in copy operation without wildcard
YingyingPeng22 opened a new pull request #13919:
URL: https://github.com/apache/airflow/pull/13919
Hi @potiuk,
This PR fix GCS copy operation without wildcard and rename destination object according to source file name.
related: #9803
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-775593856
Nice! What's next :)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-770825578
[The Workflow run](https://github.com/apache/airflow/actions/runs/527624528) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-774514968
[The Workflow run](https://github.com/apache/airflow/actions/runs/543595743) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-771073502
Those are nice examples, but what I really meant was to add those documentation in the Howto Guides. We have rather good How-to guides which are part of the official documentation and it would be great to update them there :).
https://github.com/apache/airflow/blob/master/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gcs.rst
The usual way we do it, we update example dags and then include pieces of the example dags in the howto guides (see the .rst file for examples on how this is done). The example dags are also executable and can be run as part of system tests, so this makes sure we can rely on the documentation being correct as well!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] YingyingPeng22 commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
YingyingPeng22 commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-774540578
Hi @potiuk,
Thank you for your comments! I just made changes on the example DAG and the how-to guide. Please let me know what do you think. :)
Yingying
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-775593807
Awesome work, congrats on your first merged pull request!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] YingyingPeng22 removed a comment on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
YingyingPeng22 removed a comment on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-770464319
Hi @potiuk,
I just added 2 examples and more detail explanations under function def _copy_source_without_wildcard. I am not sure if it is better to put them right below the function or add them in specifications below class GCSToGCSOperator. (line 31).
Any comments are most welcome. :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #13919:
URL: https://github.com/apache/airflow/pull/13919
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] YingyingPeng22 commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
YingyingPeng22 commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-770464319
Hi @potiuk,
I just added 2 examples and more detail explanations under function def _copy_source_without_wildcard. I am not sure if it is better to put them right below the function or add them in specifications below class GCSToGCSOperator. (line 31).
Any comments are most welcome. :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-774515043
[The Workflow run](https://github.com/apache/airflow/actions/runs/543535029) is cancelling this PR. Building image for the PR has been cancelled
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] YingyingPeng22 commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
YingyingPeng22 commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-770981857
Hi @potiuk,
I just added 2 examples and more detail explanations under function def _copy_source_without_wildcard. I am not sure if it is better to put them right below the function or add them in specifications below class GCSToGCSOperator. (line 31).
Any comments are most welcome. :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #13919: #9803 fix bug in copy operation without wildcard
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #13919:
URL: https://github.com/apache/airflow/pull/13919#issuecomment-770224298
While I think using the destination object is not a good idea in this case as the target for all sources (obviously) it would be great to add some examples and describtion/HowToGuide for the operator describing all the various options and how it behaves, as it is not obvious.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org