You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2022/07/12 10:56:03 UTC

[airflow] branch main updated: Migrate Google example gcs_to_gdrive to new design AIP-47 (#24949)

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 626d9db290 Migrate Google example gcs_to_gdrive to new design AIP-47 (#24949)
626d9db290 is described below

commit 626d9db2908563c4b7675db5de2cb1e3acde82e9
Author: Chenglong Yan <al...@gmail.com>
AuthorDate: Tue Jul 12 18:55:57 2022 +0800

    Migrate Google example gcs_to_gdrive to new design AIP-47 (#24949)
    
    related: #22447, #22430
---
 .../operators/transfer/gcs_to_gdrive.rst                  |  6 +++---
 .../providers/google/cloud/gcs}/example_gcs_to_gdrive.py  | 15 ++++++++++++---
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gdrive.rst b/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gdrive.rst
index f964845dda..9ed1cd58a8 100644
--- a/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gdrive.rst
+++ b/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gdrive.rst
@@ -47,7 +47,7 @@ Copy single files
 
 The following Operator would copy a single file.
 
-.. exampleinclude:: /../../airflow/providers/google/suite/example_dags/example_gcs_to_gdrive.py
+.. exampleinclude:: /../../tests/system/providers/google/cloud/gcs/example_gcs_to_gdrive.py
     :language: python
     :dedent: 4
     :start-after: [START howto_operator_gcs_to_gdrive_copy_single_file]
@@ -58,7 +58,7 @@ Copy multiple files
 
 The following Operator would copy all the multiples files (i.e. using wildcard).
 
-.. exampleinclude:: /../../airflow/providers/google/suite/example_dags/example_gcs_to_gdrive.py
+.. exampleinclude:: /../../tests/system/providers/google/cloud/gcs/example_gcs_to_gdrive.py
     :language: python
     :dedent: 4
     :start-after: [START howto_operator_gcs_to_gdrive_copy_files]
@@ -70,7 +70,7 @@ Move files
 Using the ``move_object`` parameter allows you to move the files. After copying the file to Google Drive,
 the original file from the bucket is deleted.
 
-.. exampleinclude:: /../../airflow/providers/google/suite/example_dags/example_gcs_to_gdrive.py
+.. exampleinclude:: /../../tests/system/providers/google/cloud/gcs/example_gcs_to_gdrive.py
     :language: python
     :dedent: 4
     :start-after: [START howto_operator_gcs_to_gdrive_move_files]
diff --git a/airflow/providers/google/suite/example_dags/example_gcs_to_gdrive.py b/tests/system/providers/google/cloud/gcs/example_gcs_to_gdrive.py
similarity index 86%
rename from airflow/providers/google/suite/example_dags/example_gcs_to_gdrive.py
rename to tests/system/providers/google/cloud/gcs/example_gcs_to_gdrive.py
index e4602c464c..06773adecc 100644
--- a/airflow/providers/google/suite/example_dags/example_gcs_to_gdrive.py
+++ b/tests/system/providers/google/cloud/gcs/example_gcs_to_gdrive.py
@@ -24,14 +24,17 @@ from datetime import datetime
 from airflow import models
 from airflow.providers.google.suite.transfers.gcs_to_gdrive import GCSToGoogleDriveOperator
 
+ENV_ID = os.environ.get("SYSTEM_TESTS_ENV_ID")
+DAG_ID = "example_gcs_to_gdrive"
+
 GCS_TO_GDRIVE_BUCKET = os.environ.get("GCS_TO_DRIVE_BUCKET", "example-object")
 
 with models.DAG(
-    "example_gcs_to_gdrive",
-    schedule_interval=None,  # Override to match your needs,
+    DAG_ID,
+    schedule_interval="@once",
     start_date=datetime(2021, 1, 1),
     catchup=False,
-    tags=['example'],
+    tags=['example', 'gcs'],
 ) as dag:
     # [START howto_operator_gcs_to_gdrive_copy_single_file]
     copy_single_file = GCSToGoogleDriveOperator(
@@ -57,3 +60,9 @@ with models.DAG(
         move_object=True,
     )
     # [END howto_operator_gcs_to_gdrive_move_files]
+
+
+from tests.system.utils import get_test_run  # noqa: E402
+
+# Needed to run the example DAG with pytest (see: tests/system/README.md#run_via_pytest)
+test_run = get_test_run(dag)