You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/20 03:11:51 UTC
[GitHub] [airflow] hankehly opened a new pull request, #25845: pdate code examples from "classic" operators to taskflow
hankehly opened a new pull request, #25845:
URL: https://github.com/apache/airflow/pull/25845
Related: #25319
(WIP)
To view the updated docs in a browser:
1. Build the docs
```
breeze build-docs --docs-only --package-filter apache-airflow
```
2. Open `docs/_build/docs/apache-airflow/latest/index.html` in your browser
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #25845:
URL: https://github.com/apache/airflow/pull/25845#issuecomment-1223502131
Need to fix static check errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #25845:
URL: https://github.com/apache/airflow/pull/25845#discussion_r950781600
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -22,8 +22,11 @@
PythonOperator
==============
-Use the :class:`~airflow.operators.python.PythonOperator` to execute
-Python callables.
+Use the ``@task`` decorator to execute Python callables.
+
+.. warning::
+ Previous versions of Airflow required using :class:`~airflow.operators.python.PythonOperator`
+ to execute Python callables. The ``@task`` decorater is now the recommended method.
Review Comment:
The last version to _require_ using PythonOperator was 1.10.10; I’d probably rewrite this to
> The `@task` decorator is recommended to the more “classic” `PythonOperator` approach to execute Python callables.
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -22,8 +22,11 @@
PythonOperator
==============
-Use the :class:`~airflow.operators.python.PythonOperator` to execute
-Python callables.
+Use the ``@task`` decorator to execute Python callables.
+
+.. warning::
+ Previous versions of Airflow required using :class:`~airflow.operators.python.PythonOperator`
+ to execute Python callables. The ``@task`` decorater is now the recommended method.
Review Comment:
The last version to _require_ using PythonOperator was 1.10.10; I’d probably rewrite this to
> The `@task` decorator is recommended over the more “classic” `PythonOperator` approach to execute Python callables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a diff in pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
kaxil commented on code in PR #25845:
URL: https://github.com/apache/airflow/pull/25845#discussion_r952563656
##########
docs/apache-airflow/upgrading-from-1-10/index.rst:
##########
@@ -924,6 +929,23 @@ instead. On top of that, a new dag.callback_exceptions counter metric has
been added to help better monitor callback exceptions.
+Migrating to TaskFlow API
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Airflow 2.0 introduced the TaskFlow API to simplify the declaration of Python callable tasks.
+Users are encouraged to replace classic operators with their TaskFlow decorator alternatives.
+For details, see :doc:`/tutorial_taskflow_api`.
+
+============================= ============================================
+Classic Operator TaskFlow Decorator
+============================= ============================================
+``PythonOperator`` ``@task`` (short for ``@task.python``)
+``PythonVirtualenvOperator`` ``@task.virtualenv``
+``BranchPythonOperator`` ``@task.branch``
+``DockerOperator`` ``@task.docker``
+============================= ============================================
Review Comment:
it was added in https://github.com/apache/airflow/pull/25663
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #25845:
URL: https://github.com/apache/airflow/pull/25845#discussion_r950795366
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -22,8 +22,11 @@
PythonOperator
==============
-Use the :class:`~airflow.operators.python.PythonOperator` to execute
-Python callables.
+Use the ``@task`` decorator to execute Python callables.
+
+.. warning::
+ Previous versions of Airflow required using :class:`~airflow.operators.python.PythonOperator`
+ to execute Python callables. The ``@task`` decorater is now the recommended method.
Review Comment:
Thanks. Please see [`fc66a4e`](https://github.com/apache/airflow/pull/25845/commits/fc66a4efb5d418e0aaee583406aa1fe9b3421621)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #25845:
URL: https://github.com/apache/airflow/pull/25845#discussion_r950645198
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -22,8 +22,11 @@
PythonOperator
Review Comment:
We could go a step further and retitle this page "@task"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
hankehly commented on PR #25845:
URL: https://github.com/apache/airflow/pull/25845#issuecomment-1221227051
(waiting for checks to pass before requesting review)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #25845:
URL: https://github.com/apache/airflow/pull/25845#discussion_r953323431
##########
docs/apache-airflow/upgrading-from-1-10/index.rst:
##########
@@ -924,6 +929,23 @@ instead. On top of that, a new dag.callback_exceptions counter metric has
been added to help better monitor callback exceptions.
+Migrating to TaskFlow API
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Airflow 2.0 introduced the TaskFlow API to simplify the declaration of Python callable tasks.
+Users are encouraged to replace classic operators with their TaskFlow decorator alternatives.
+For details, see :doc:`/tutorial_taskflow_api`.
+
+============================= ============================================
+Classic Operator TaskFlow Decorator
+============================= ============================================
+``PythonOperator`` ``@task`` (short for ``@task.python``)
+``PythonVirtualenvOperator`` ``@task.virtualenv``
+``BranchPythonOperator`` ``@task.branch``
+``DockerOperator`` ``@task.docker``
+============================= ============================================
Review Comment:
@kaxil Added in [dfd4327](https://github.com/apache/airflow/pull/25845/commits/dfd43274474c85bd76bf301254e1c1e2d7c7db49)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
hankehly commented on PR #25845:
URL: https://github.com/apache/airflow/pull/25845#issuecomment-1223649671
@uranusjr
Static checks fixed in https://github.com/apache/airflow/pull/25845/commits/fad7585dcbfcd3eaf136a7db70f25b157e90cae6
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #25845:
URL: https://github.com/apache/airflow/pull/25845
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #25845:
URL: https://github.com/apache/airflow/pull/25845#discussion_r953259832
##########
docs/apache-airflow/upgrading-from-1-10/index.rst:
##########
@@ -924,6 +929,23 @@ instead. On top of that, a new dag.callback_exceptions counter metric has
been added to help better monitor callback exceptions.
+Migrating to TaskFlow API
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Airflow 2.0 introduced the TaskFlow API to simplify the declaration of Python callable tasks.
+Users are encouraged to replace classic operators with their TaskFlow decorator alternatives.
+For details, see :doc:`/tutorial_taskflow_api`.
+
+============================= ============================================
+Classic Operator TaskFlow Decorator
+============================= ============================================
+``PythonOperator`` ``@task`` (short for ``@task.python``)
+``PythonVirtualenvOperator`` ``@task.virtualenv``
+``BranchPythonOperator`` ``@task.branch``
+``DockerOperator`` ``@task.docker``
+============================= ============================================
Review Comment:
Thanks, let's add that in
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a diff in pull request #25845: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
kaxil commented on code in PR #25845:
URL: https://github.com/apache/airflow/pull/25845#discussion_r952563181
##########
docs/apache-airflow/upgrading-from-1-10/index.rst:
##########
@@ -924,6 +929,23 @@ instead. On top of that, a new dag.callback_exceptions counter metric has
been added to help better monitor callback exceptions.
+Migrating to TaskFlow API
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Airflow 2.0 introduced the TaskFlow API to simplify the declaration of Python callable tasks.
+Users are encouraged to replace classic operators with their TaskFlow decorator alternatives.
+For details, see :doc:`/tutorial_taskflow_api`.
+
+============================= ============================================
+Classic Operator TaskFlow Decorator
+============================= ============================================
+``PythonOperator`` ``@task`` (short for ``@task.python``)
+``PythonVirtualenvOperator`` ``@task.virtualenv``
+``BranchPythonOperator`` ``@task.branch``
+``DockerOperator`` ``@task.docker``
+============================= ============================================
Review Comment:
We also have ``@task.kubernetes`` now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org