You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/09 17:26:59 UTC
[GitHub] [airflow] andrewgodwin opened a new pull request #15310: Add picture and examples for Edge Labels
andrewgodwin opened a new pull request #15310:
URL: https://github.com/apache/airflow/pull/15310
This builds on the main feature which landed in 19b74fd.
It adds a picture to the docs, a new example DAG that creates that picture, and adds Label into the branch operator example too as that is likely where it is most useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15310: Add picture and examples for Edge Labels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15310:
URL: https://github.com/apache/airflow/pull/15310#issuecomment-816994429
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #15310: Add picture and examples for Edge Labels
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #15310:
URL: https://github.com/apache/airflow/pull/15310
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #15310: Add picture and examples for Edge Labels
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #15310:
URL: https://github.com/apache/airflow/pull/15310#discussion_r610856590
##########
File path: airflow/example_dags/example_branch_labels.py
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example DAG demonstrating the usage of labels with different branches.
+"""
+
+from airflow import DAG
+from airflow.operators.dummy import DummyOperator
+from airflow.utils.dates import days_ago
+from airflow.utils.edgemodifier import Label
+
+with DAG("test_three", schedule_interval="@daily", start_date=days_ago(2)) as dag:
Review comment:
```suggestion
with DAG("example_branch_labels", schedule_interval="@daily", start_date=days_ago(2)) as dag:
```
Since this will appear in the Webserver sorted by dag id
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] andrewgodwin commented on a change in pull request #15310: Add picture and examples for Edge Labels
Posted by GitBox <gi...@apache.org>.
andrewgodwin commented on a change in pull request #15310:
URL: https://github.com/apache/airflow/pull/15310#discussion_r610881823
##########
File path: airflow/example_dags/example_branch_labels.py
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example DAG demonstrating the usage of labels with different branches.
+"""
+
+from airflow import DAG
+from airflow.operators.dummy import DummyOperator
+from airflow.utils.dates import days_ago
+from airflow.utils.edgemodifier import Label
+
+with DAG("test_three", schedule_interval="@daily", start_date=days_ago(2)) as dag:
Review comment:
Looks like that made it out of my test directory badly :) I'll update it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #15310: Add picture and examples for Edge Labels
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #15310:
URL: https://github.com/apache/airflow/pull/15310#discussion_r610926845
##########
File path: airflow/example_dags/example_branch_labels.py
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example DAG demonstrating the usage of labels with different branches.
+"""
+
+from airflow import DAG
+from airflow.operators.dummy import DummyOperator
+from airflow.utils.dates import days_ago
+from airflow.utils.edgemodifier import Label
+
+with DAG("test_three", schedule_interval="@daily", start_date=days_ago(2)) as dag:
+
+ ingest = DummyOperator(task_id="ingest")
+ analyse = DummyOperator(task_id="analyze")
+ check = DummyOperator(task_id="check_integrity")
+ describe = DummyOperator(task_id="describe_integrity")
+ error = DummyOperator(task_id="email_error")
+ save = DummyOperator(task_id="save")
+ report = DummyOperator(task_id="report")
+
+ ingest >> analyse >> check
+ check >> Label("No errors") >> save >> report # pylint: disable=W0106
+ check >> Label("Errors found") >> describe >> error >> report # pylint: disable=W0106
Review comment:
I support disabling it for the examples directory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] andrewgodwin commented on a change in pull request #15310: Add picture and examples for Edge Labels
Posted by GitBox <gi...@apache.org>.
andrewgodwin commented on a change in pull request #15310:
URL: https://github.com/apache/airflow/pull/15310#discussion_r610882051
##########
File path: airflow/example_dags/example_branch_labels.py
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example DAG demonstrating the usage of labels with different branches.
+"""
+
+from airflow import DAG
+from airflow.operators.dummy import DummyOperator
+from airflow.utils.dates import days_ago
+from airflow.utils.edgemodifier import Label
+
+with DAG("test_three", schedule_interval="@daily", start_date=days_ago(2)) as dag:
+
+ ingest = DummyOperator(task_id="ingest")
+ analyse = DummyOperator(task_id="analyze")
+ check = DummyOperator(task_id="check_integrity")
+ describe = DummyOperator(task_id="describe_integrity")
+ error = DummyOperator(task_id="email_error")
+ save = DummyOperator(task_id="save")
+ report = DummyOperator(task_id="report")
+
+ ingest >> analyse >> check
+ check >> Label("No errors") >> save >> report # pylint: disable=W0106
+ check >> Label("Errors found") >> describe >> error >> report # pylint: disable=W0106
Review comment:
Good idea, I'll do that. I would also argue that maybe we should just disable that message for at least the examples directory, given how Airflow dags work!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #15310: Add picture and examples for Edge Labels
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #15310:
URL: https://github.com/apache/airflow/pull/15310#discussion_r610859099
##########
File path: airflow/example_dags/example_branch_labels.py
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example DAG demonstrating the usage of labels with different branches.
+"""
+
+from airflow import DAG
+from airflow.operators.dummy import DummyOperator
+from airflow.utils.dates import days_ago
+from airflow.utils.edgemodifier import Label
+
+with DAG("test_three", schedule_interval="@daily", start_date=days_ago(2)) as dag:
+
+ ingest = DummyOperator(task_id="ingest")
+ analyse = DummyOperator(task_id="analyze")
+ check = DummyOperator(task_id="check_integrity")
+ describe = DummyOperator(task_id="describe_integrity")
+ error = DummyOperator(task_id="email_error")
+ save = DummyOperator(task_id="save")
+ report = DummyOperator(task_id="report")
+
+ ingest >> analyse >> check
+ check >> Label("No errors") >> save >> report # pylint: disable=W0106
+ check >> Label("Errors found") >> describe >> error >> report # pylint: disable=W0106
Review comment:
Should we use pylint messages instead of codes, that would help others in reading the code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org