You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/09/28 02:05:13 UTC
[GitHub] [airflow] josh-fell opened a new pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
josh-fell opened a new pull request #18562:
URL: https://github.com/apache/airflow/pull/18562
Related: #9415
This PR aims to replace the use of PythonOperator tasks for the TaskFlow API in several core example DAGs.
Additionally, there are instances of replacing `trigger_rule` values with the appropriate `TriggerRule` attr instead of the literal string, removing an unnecessary `dag` arg in _example_skip_dag_, and replacing `PythonOperator` in _example_complex_ for `BashOperator` (since it was the only task that wasn't a `BashOperator` and using a `PythonOperator` task really wasn't adding any value to the example).
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932341920
There's a test that looks flaky and related to this PR: https://github.com/apache/airflow/pull/18562/checks?check_run_id=3735447666
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr closed pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
uranusjr closed pull request #18562:
URL: https://github.com/apache/airflow/pull/18562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr closed pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
uranusjr closed pull request #18562:
URL: https://github.com/apache/airflow/pull/18562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-942199839
It did :).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932587632
Looks like there is differing sorting behavior between Postgres and other backend databases for this test. The `TaskInstance` rows returned aren't always in the same order especially between database types. This is an issue because the assertion result is relative to a certain order of `TaskInstance` rows.
In the failing test, these are the states of the `TaskInstances`:
```python
tis = [
TaskInstance(dag.get_task("runme_0"), run_id=dr.run_id, state="success"),
TaskInstance(dag.get_task("runme_1"), run_id=dr.run_id, state="failed"),
]
```
Running this test and printing the assertion list comprehension with multiple backends, you can see that Postgres retrieves the data in a different order than MySQL and SQLite. The `TaskInstance` "example_bash_operator.runme_1" is sorted before "example_bash_operator.runme_0":
``` shell
**PostgreSQL 9.6**
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 21:00:12,320] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:00:12,630] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:00:12,641] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>, <TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>]
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 21:00:13,373] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:00:13,699] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:00:13,709] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>, <TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>]
**SQLite**
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 20:56:52,545] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 20:56:52,930] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 20:56:52,942] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 20:56:53,677] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 20:56:54,016] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 20:56:54,026] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
*MySQL 5.7**
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 21:06:09,569] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:06:09,891] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:06:09,901] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 21:06:10,632] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:06:10,965] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:06:10,977] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
```
I can create an issue to update the `test_set_dag_runs_action` test such that the rows retrieved as sorted consistently or explicitly checking the state of individual `TaskInstance` states. WDYT? Either solution should fix the flaky test IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#discussion_r720362346
##########
File path: airflow/example_dags/example_trigger_target_dag.py
##########
@@ -24,11 +24,12 @@
from datetime import datetime
from airflow import DAG
+from airflow.decorators import task
from airflow.operators.bash import BashOperator
-from airflow.operators.python import PythonOperator
-def run_this_func(dag_run):
+@task(task_id="run_this")
+def run_this_func(dag_run=None):
Review comment:
I see, good point. I feel it's better to show that you don't need to explicitly pass in the `dag_run` object (any context object for that matter) when using the TaskFlow API (hoping it's another example that makes the TaskFlow API more approachable). I think updating the access method would be a better move.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932598462
Ops, the test is using example_bash_operator and we didn't work on that in this PR. Strange!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#discussion_r720349141
##########
File path: airflow/example_dags/example_trigger_target_dag.py
##########
@@ -24,11 +24,12 @@
from datetime import datetime
from airflow import DAG
+from airflow.decorators import task
from airflow.operators.bash import BashOperator
-from airflow.operators.python import PythonOperator
-def run_this_func(dag_run):
+@task(task_id="run_this")
+def run_this_func(dag_run=None):
Review comment:
Actually this is supposed to be used in tandem with the `example_trigger_controller_dag` DAG which triggers this DAG based on the docstring:
```
"""
Example usage of the TriggerDagRunOperator. This example holds 2 DAGs:
1. 1st DAG (example_trigger_controller_dag) holds a TriggerDagRunOperator, which will trigger the 2nd DAG
2. 2nd DAG (example_trigger_target_dag) which will be triggered by the TriggerDagRunOperator in the 1st DAG
"""
```
Verified the "controller" DAG has the `TriggerDagRunOperator` sending a `conf` arg:
```python
trigger = TriggerDagRunOperator(
task_id="test_trigger_dagrun",
trigger_dag_id="example_trigger_target_dag", # Ensure this equals the dag_id of the DAG to trigger
conf={"message": "Hello World"},
)
```
Do you think it's still worth adding a simple check on `dag_run.conf["message"]`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-941845645
Rebasing. Looks like #18853 fixes the failing `tests/sensors/test_external_task_sensor.py` test for Postgres13, Py3.8.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy merged pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
ephraimbuddy merged pull request #18562:
URL: https://github.com/apache/airflow/pull/18562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-942199839
It did :).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932350405
> There's a test that looks flaky and related to this PR: https://github.com/apache/airflow/pull/18562/checks?check_run_id=3735447666
👍 I'll check it out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932085164
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell edited a comment on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell edited a comment on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932739457
#18618 actually fixed this test a few days ago so I'll rebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932738031
> Ops, the test is using example_bash_operator and we didn't work on that in this PR. Strange!
Very. And I've spot-checked some other PRs and the test is passing. I'll take another look but other than data sorting another root cause is escaping me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#discussion_r720101808
##########
File path: airflow/example_dags/example_trigger_target_dag.py
##########
@@ -24,11 +24,12 @@
from datetime import datetime
from airflow import DAG
+from airflow.decorators import task
from airflow.operators.bash import BashOperator
-from airflow.operators.python import PythonOperator
-def run_this_func(dag_run):
+@task(task_id="run_this")
+def run_this_func(dag_run=None):
Review comment:
This would fail if there's no dag_run because of the print statement below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr closed pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
uranusjr closed pull request #18562:
URL: https://github.com/apache/airflow/pull/18562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-941845645
Rebasing. Looks like #18853 fixes the failing `tests/sensors/test_external_task_sensor.py` test for Postgres13, Py3.8.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell edited a comment on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell edited a comment on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932587632
Looks like there is differing sorting behavior between Postgres and other backend databases for this test. The `TaskInstance` rows returned aren't always in the same order especially between database types. This is an issue because the assertion result is relative to a certain order of `TaskInstance` rows.
In the failing `test_set_dag_runs_action ` test with Postgres, these are the states of the `TaskInstances`:
```python
tis = [
TaskInstance(dag.get_task("runme_0"), run_id=dr.run_id, state="success"),
TaskInstance(dag.get_task("runme_1"), run_id=dr.run_id, state="failed"),
]
```
Running this test and printing the assertion list comprehension with multiple backends, you can see that Postgres retrieves the data in a different order than MySQL and SQLite. The `TaskInstance` "example_bash_operator.runme_1" is sorted before "example_bash_operator.runme_0":
``` shell
**PostgreSQL 9.6**
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 21:00:12,320] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:00:12,630] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:00:12,641] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>, <TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>]
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 21:00:13,373] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:00:13,699] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:00:13,709] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>, <TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>]
**SQLite**
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 20:56:52,545] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 20:56:52,930] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 20:56:52,942] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 20:56:53,677] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 20:56:54,016] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 20:56:54,026] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
*MySQL 5.7**
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 21:06:09,569] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:06:09,891] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:06:09,901] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 21:06:10,632] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags
[2021-10-01 21:06:10,965] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
[2021-10-01 21:06:10,977] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>]
```
I can create an issue to update the `test_set_dag_runs_action` test such that the rows retrieved as sorted consistently or explicitly checking the state of individual `TaskInstance` states. WDYT? Either solution should fix the flaky test IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy merged pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
ephraimbuddy merged pull request #18562:
URL: https://github.com/apache/airflow/pull/18562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#discussion_r720248889
##########
File path: airflow/example_dags/example_trigger_target_dag.py
##########
@@ -24,11 +24,12 @@
from datetime import datetime
from airflow import DAG
+from airflow.decorators import task
from airflow.operators.bash import BashOperator
-from airflow.operators.python import PythonOperator
-def run_this_func(dag_run):
+@task(task_id="run_this")
+def run_this_func(dag_run=None):
Review comment:
Yes it sure would. Thanks for the catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
josh-fell commented on pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#issuecomment-932739457
#18618 Actually fixed this test a few days ago so I'll rebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #18562: Updating core example DAGs to use TaskFlow API where applicable
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #18562:
URL: https://github.com/apache/airflow/pull/18562#discussion_r720354001
##########
File path: airflow/example_dags/example_trigger_target_dag.py
##########
@@ -24,11 +24,12 @@
from datetime import datetime
from airflow import DAG
+from airflow.decorators import task
from airflow.operators.bash import BashOperator
-from airflow.operators.python import PythonOperator
-def run_this_func(dag_run):
+@task(task_id="run_this")
+def run_this_func(dag_run=None):
Review comment:
From a syntax standpoint, the dag_run should be a required value if we access its attribute as we did. I would suggest we change it but I don't have strong opinion on it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org