You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/17 10:04:48 UTC
[GitHub] [airflow] chenglongyan opened a new pull request, #25761: Migrate Google example gcp_transfer to new design AIP-47
chenglongyan opened a new pull request, #25761:
URL: https://github.com/apache/airflow/pull/25761
related: #22447, #22430
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a diff in pull request #25761: Migrate Google example gcp_transfer to new design AIP-47
Posted by GitBox <gi...@apache.org>.
josh-fell commented on code in PR #25761:
URL: https://github.com/apache/airflow/pull/25761#discussion_r948060016
##########
tests/system/providers/google/cloud/transfers/example_cloud_storage_transfer_service_gcp.py:
##########
@@ -142,13 +145,28 @@
task_id="delete_transfer_from_gcp_job",
job_name="{{task_instance.xcom_pull('create_transfer')['name']}}",
project_id=GCP_PROJECT_ID,
+ trigger_rule=TriggerRule.ALL_DONE,
)
- chain(
Review Comment:
Just curious why not keep `chain()` here? You can still add the comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan closed pull request #25761: Migrate Google example gcp_transfer to new design AIP-47
Posted by GitBox <gi...@apache.org>.
chenglongyan closed pull request #25761: Migrate Google example gcp_transfer to new design AIP-47
URL: https://github.com/apache/airflow/pull/25761
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bhirsz commented on a diff in pull request #25761: Migrate Google example gcp_transfer to new design AIP-47
Posted by GitBox <gi...@apache.org>.
bhirsz commented on code in PR #25761:
URL: https://github.com/apache/airflow/pull/25761#discussion_r953605201
##########
tests/system/providers/google/cloud/transfers/example_cloud_storage_transfer_service_gcp.py:
##########
@@ -100,12 +103,12 @@
# [END howto_operator_gcp_transfer_update_job_body]
with models.DAG(
- "example_gcp_transfer",
+ DAG_ID,
+ schedule='@once',
start_date=datetime(2021, 1, 1),
catchup=False,
- tags=["example"],
+ tags=["example", "gcp"],
) as dag:
-
create_transfer = CloudDataTransferServiceCreateJobOperator(
Review Comment:
In old design, pytest class, we're creating bucket before test:
```
GoogleSystemTest.create_gcs_bucket(GCP_TRANSFER_SECOND_TARGET_BUCKET, location="asia-east1")
GoogleSystemTest.create_gcs_bucket(GCP_TRANSFER_FIRST_TARGET_BUCKET)
```
We need to create those buckets here as well (and remove them with trigger ALL_DONE) - refer to other system tests to see how it could be done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan commented on pull request #25761: Migrate Google example gcp_transfer to new design AIP-47
Posted by GitBox <gi...@apache.org>.
chenglongyan commented on PR #25761:
URL: https://github.com/apache/airflow/pull/25761#issuecomment-1225941030
I feel like this pr should be closed, I didn't consider the old system test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan commented on a diff in pull request #25761: Migrate Google example gcp_transfer to new design AIP-47
Posted by GitBox <gi...@apache.org>.
chenglongyan commented on code in PR #25761:
URL: https://github.com/apache/airflow/pull/25761#discussion_r950982191
##########
tests/system/providers/google/cloud/transfers/example_cloud_storage_transfer_service_gcp.py:
##########
@@ -142,13 +145,28 @@
task_id="delete_transfer_from_gcp_job",
job_name="{{task_instance.xcom_pull('create_transfer')['name']}}",
project_id=GCP_PROJECT_ID,
+ trigger_rule=TriggerRule.ALL_DONE,
)
- chain(
Review Comment:
Just to be consistent with the AIP-47 example
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org