You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2022/06/03 16:08:03 UTC
[airflow] branch main updated: Migrate Postgres example DAGs to new design #22458 (#24148)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new c60bb9edc0 Migrate Postgres example DAGs to new design #22458 (#24148)
c60bb9edc0 is described below
commit c60bb9edc0c9b55a2824eae879af8a4a90ccdd2d
Author: chethanuk-plutoflume <ch...@tessian.com>
AuthorDate: Fri Jun 3 17:07:56 2022 +0100
Migrate Postgres example DAGs to new design #22458 (#24148)
* Migrate Postgres example DAGs to new design #22458
* Fix static checks
---
.../providers/postgres/example_dags/__init__.py | 16 ----------------
docs/apache-airflow-providers-postgres/index.rst | 2 +-
.../operators/postgres_operator_howto_guide.rst | 6 +++---
.../system/providers/postgres}/example_postgres.py | 22 +++++++++++++++++++---
4 files changed, 23 insertions(+), 23 deletions(-)
diff --git a/airflow/providers/postgres/example_dags/__init__.py b/airflow/providers/postgres/example_dags/__init__.py
deleted file mode 100644
index 13a83393a9..0000000000
--- a/airflow/providers/postgres/example_dags/__init__.py
+++ /dev/null
@@ -1,16 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
diff --git a/docs/apache-airflow-providers-postgres/index.rst b/docs/apache-airflow-providers-postgres/index.rst
index e99bf48e82..376fe722ed 100644
--- a/docs/apache-airflow-providers-postgres/index.rst
+++ b/docs/apache-airflow-providers-postgres/index.rst
@@ -39,7 +39,7 @@ Content
:maxdepth: 1
:caption: Resources
- Example DAGs <https://github.com/apache/airflow/tree/main/airflow/providers/postgres/example_dags>
+ Example DAGs <https://github.com/apache/airflow/tree/main/tests/system/providers/postgres>
.. toctree::
:maxdepth: 1
diff --git a/docs/apache-airflow-providers-postgres/operators/postgres_operator_howto_guide.rst b/docs/apache-airflow-providers-postgres/operators/postgres_operator_howto_guide.rst
index 2a5ee5a333..790d02caec 100644
--- a/docs/apache-airflow-providers-postgres/operators/postgres_operator_howto_guide.rst
+++ b/docs/apache-airflow-providers-postgres/operators/postgres_operator_howto_guide.rst
@@ -42,7 +42,7 @@ Creating a Postgres database table
The code snippets below are based on Airflow-2.0
-.. exampleinclude:: /../../airflow/providers/postgres/example_dags/example_postgres.py
+.. exampleinclude:: /../../tests/system/providers/postgres/example_postgres.py
:language: python
:start-after: [START postgres_operator_howto_guide]
:end-before: [END postgres_operator_howto_guide_create_pet_table]
@@ -160,7 +160,7 @@ Passing Server Configuration Parameters into PostgresOperator
PostgresOperator provides the optional ``runtime_parameters`` attribute which makes it possible to set
the `server configuration parameter values <https://www.postgresql.org/docs/current/runtime-config-client.html>`_ for the SQL request during runtime.
-.. exampleinclude:: /../../airflow/providers/postgres/example_dags/example_postgres.py
+.. exampleinclude:: /../../tests/system/providers/postgres/example_postgres.py
:language: python
:start-after: [START postgres_operator_howto_guide_get_birth_date]
:end-before: [END postgres_operator_howto_guide_get_birth_date]
@@ -171,7 +171,7 @@ The complete Postgres Operator DAG
When we put everything together, our DAG should look like this:
-.. exampleinclude:: /../../airflow/providers/postgres/example_dags/example_postgres.py
+.. exampleinclude:: /../../tests/system/providers/postgres/example_postgres.py
:language: python
:start-after: [START postgres_operator_howto_guide]
:end-before: [END postgres_operator_howto_guide]
diff --git a/airflow/providers/postgres/example_dags/example_postgres.py b/tests/system/providers/postgres/example_postgres.py
similarity index 86%
rename from airflow/providers/postgres/example_dags/example_postgres.py
rename to tests/system/providers/postgres/example_postgres.py
index 18a95d84d3..b7d6c0d0a8 100644
--- a/airflow/providers/postgres/example_dags/example_postgres.py
+++ b/tests/system/providers/postgres/example_postgres.py
@@ -14,18 +14,23 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-
-# [START postgres_operator_howto_guide]
import datetime
+import os
from airflow import DAG
from airflow.providers.postgres.operators.postgres import PostgresOperator
+# [START postgres_operator_howto_guide]
+
+
# create_pet_table, populate_pet_table, get_all_pets, and get_birth_date are examples of tasks created by
# instantiating the Postgres Operator
+ENV_ID = os.environ.get("SYSTEM_TESTS_ENV_ID")
+DAG_ID = "postgres_operator_dag"
+
with DAG(
- dag_id="postgres_operator_dag",
+ dag_id=DAG_ID,
start_date=datetime.datetime(2020, 2, 2),
schedule_interval="@once",
catchup=False,
@@ -72,3 +77,14 @@ with DAG(
create_pet_table >> populate_pet_table >> get_all_pets >> get_birth_date
# [END postgres_operator_howto_guide]
+
+ from tests.system.utils.watcher import watcher
+
+ # This test needs watcher in order to properly mark success/failure
+ # when "tearDown" task with trigger rule is part of the DAG
+ list(dag.tasks) >> watcher()
+
+from tests.system.utils import get_test_run # noqa: E402
+
+# Needed to run the example DAG with pytest (see: tests/system/README.md#run_via_pytest)
+test_run = get_test_run(dag)