You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/11 05:00:20 UTC
[GitHub] [airflow] hankehly opened a new pull request, #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
hankehly opened a new pull request, #25657:
URL: https://github.com/apache/airflow/pull/25657
Related: #25319
This PR replaces uses of `PythonOperator` with the `@task` TaskFlow decorator in faq, best practices and "index" documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #25657:
URL: https://github.com/apache/airflow/pull/25657#discussion_r946224940
##########
docs/apache-airflow/best-practices.rst:
##########
@@ -135,26 +135,21 @@ Bad example:
catchup=False,
tags=["example"],
) as dag:
-
+ @task()
def print_array():
Review Comment:
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr merged pull request #25657: Update code examples from "classic" operators to taskflow
Posted by GitBox <gi...@apache.org>.
uranusjr merged PR #25657:
URL: https://github.com/apache/airflow/pull/25657
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
Posted by GitBox <gi...@apache.org>.
josh-fell commented on PR #25657:
URL: https://github.com/apache/airflow/pull/25657#issuecomment-1218126797
@hankehly When you get a chance, can you address the static check errors please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #25657:
URL: https://github.com/apache/airflow/pull/25657#discussion_r946224940
##########
docs/apache-airflow/best-practices.rst:
##########
@@ -135,26 +135,21 @@ Bad example:
catchup=False,
tags=["example"],
) as dag:
-
+ @task()
def print_array():
Review Comment:
Thanks, I had a feeling someone might mention this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a diff in pull request #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
Posted by GitBox <gi...@apache.org>.
josh-fell commented on code in PR #25657:
URL: https://github.com/apache/airflow/pull/25657#discussion_r946065430
##########
docs/apache-airflow/best-practices.rst:
##########
@@ -135,26 +135,21 @@ Bad example:
catchup=False,
tags=["example"],
) as dag:
-
+ @task()
def print_array():
Review Comment:
The example should show calling `print_array()` as well. Otherwise the task _technically_ doesn't get added to the DAG. Any users, I suspect especially beginners, could copy/paste this code and not understand why the `print_array` task doesn't appear in the DAG.
##########
docs/apache-airflow/best-practices.rst:
##########
@@ -163,7 +158,7 @@ Good example:
catchup=False,
tags=["example"],
) as dag:
-
+ @task()
def print_array():
Review Comment:
Same here. Let's make sure the code snippet actually calls `print_array()`.
##########
docs/apache-airflow/index.rst:
##########
@@ -40,24 +40,27 @@ Take a look at the following snippet of code:
from datetime import datetime
from airflow import DAG
+ from airflow.decorators import task
from airflow.operators.bash import BashOperator
- from airflow.operators.python import PythonOperator
# A DAG represents a workflow, a collection of tasks
with DAG(dag_id="demo", start_date=datetime(2022, 1, 1), schedule="0 0 * * *") as dag:
# Tasks are represented as operators
hello = BashOperator(task_id="hello", bash_command="echo hello")
- airflow = PythonOperator(task_id="airflow", python_callable=lambda: print("airflow"))
+
+ @task()
+ def airflow():
+ print("airflow")
# Set dependencies between tasks
- hello >> airflow
+ hello >> airflow()
Here you see:
- A DAG named "demo", starting on Jan 1st 2022 and running once a day. A DAG is Airflow's representation of a workflow.
-- Two tasks, a BashOperator running a Bash script and a PythonOperator running a Python script
+- Two tasks, a BashOperator running a Bash script and a python function defined using the ``@task`` decorator
Review Comment:
```suggestion
- Two tasks, a BashOperator running a Bash script and a Python function defined using the ``@task`` decorator
```
Small nit here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on pull request #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
Posted by GitBox <gi...@apache.org>.
hankehly commented on PR #25657:
URL: https://github.com/apache/airflow/pull/25657#issuecomment-1215992342
@josh-fell Please see updates. I'll get started on the remaining files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #25657:
URL: https://github.com/apache/airflow/pull/25657#discussion_r947336092
##########
docs/apache-airflow/faq.rst:
##########
@@ -195,7 +191,12 @@ until ``min_file_process_interval`` is reached since DAG Parser will look for mo
)
with dag:
- t1 = PythonOperator(task_id="hello_world", python_callable=hello_world_py)
+ @task()
+ def hello_world():
+ print("Hello World")
+ print("This is DAG: {}".format(str(dag_number)))
Review Comment:
```suggestion
print(f"This is DAG: {dag_number}")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on pull request #25657: Update code examples from "classic" operators to TaskFlow decorators (faq, best practices, index)
Posted by GitBox <gi...@apache.org>.
hankehly commented on PR #25657:
URL: https://github.com/apache/airflow/pull/25657#issuecomment-1211561991
@josh-fell @potiuk
Please see this at your earliest convenience.
If it looks OK I'll go ahead and update the remaining docs in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org