[airflow] branch master updated: Updated taskflow api doc to show dependency with sensor (#13968)

[ka...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/master by this push:
     new df11a1d  Updated taskflow api doc to show dependency with sensor (#13968)
df11a1d is described below

commit df11a1d7dcc4e454b99a71805c133c3d15c197dc
Author: Vikram Koka <vi...@astronomer.io>
AuthorDate: Fri Jan 29 11:30:35 2021 -0800

    Updated taskflow api doc to show dependency with sensor (#13968)
    
    * Updated taskflow api doc to show dependency with sensor
    Updated the taskflow api tutorial document to show how to setup a
    dependency to a python-based decorated task from a classic
    FileSensor task.
---
 docs/apache-airflow/tutorial_taskflow_api.rst | 34 +++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/docs/apache-airflow/tutorial_taskflow_api.rst b/docs/apache-airflow/tutorial_taskflow_api.rst
index e389a4c..b089e03 100644
--- a/docs/apache-airflow/tutorial_taskflow_api.rst
+++ b/docs/apache-airflow/tutorial_taskflow_api.rst
@@ -177,6 +177,40 @@ is automatically set to true.
 Note, If you manually set the ``multiple_outputs`` parameter the inference is disabled and
 the parameter value is used.
 
+Adding dependencies to decorated tasks from regular tasks
+---------------------------------------------------------
+The above tutorial shows how to create dependencies between python-based tasks. However, it is
+quite possible while writing a DAG to have some pre-existing tasks such as :class:`~airflow.operators.bash.BashOperator` or :class:`~airflow.sensors.filesystem.FileSensor`
+based tasks which need to be run first before a python-based task is run.
+
+Building this dependency is shown in the code below:
+
+.. code-block:: python
+
+    @task()
+        def extract_from_file():
+        """
+        #### Extract from file task
+        A simple Extract task to get data ready for the rest of the data
+        pipeline, by reading the data from a file into a pandas dataframe
+        """
+        order_data_file = '/tmp/order_data.csv'
+        order_data_df = pd.read_csv(order_data_file)
+
+
+    file_task = FileSensor(task_id='check_file', filepath='/tmp/order_data.csv')
+    order_data = extract_from_file()
+
+    file_task >> order_data
+
+
+In the above code block, a new python-based task is defined as ``extract_from_file`` which
+reads the data from a known file location.
+In the main DAG, a new ``FileSensor`` task is defined to check for this file. Please note
+that this is a Sensor task which waits for the file.
+Finally, a dependency between this Sensor task and the python-based task is specified.
+
+
 What's Next?
 ------------