[airflow] 01/24: Adds back documentation about context usage in Python/@task (#18868)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 3714aa321a52d1130885523c73d72845f12b9009
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Oct 11 17:12:37 2021 +0200

    Adds back documentation about context usage in Python/@task (#18868)
    
    There were many questions recently along the line of
    "How do I access context from TaskFlow task". Surprisingly,
    the paragraph about accessing current context was removed from
    the "Concepts" (where it was there for Airflow 2.0.0) but was
    never added to the "TaskFlow Tutorial" where it actually belongs.
    
    Also Python Operator's description about passing context
    variables as kwargs have been removed when `provide_context`
    parameter was removed (it was only present in the docstring
    of `provide_context` and you could likely deduce this behaviour
    from several examples, but it was not mentioned anywhere.
    
    This PR adds the description with examples to the Python operator
    as well as adds similar description in TaskFlow tutorial, including
    the possibility of using `get_current_context` deep down the
    stack to retrieve the context variables even if they are not
    passed via kwargs.
    
    (cherry picked from commit d6c8730ebc737e0200091cdf58231cd3cf6afd84)
---
 docs/apache-airflow/tutorial_taskflow_api.rst | 42 +++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/docs/apache-airflow/tutorial_taskflow_api.rst b/docs/apache-airflow/tutorial_taskflow_api.rst
index 23c982b..076f48b 100644
--- a/docs/apache-airflow/tutorial_taskflow_api.rst
+++ b/docs/apache-airflow/tutorial_taskflow_api.rst
@@ -409,6 +409,48 @@ method of the Python operator.
 Current context is accessible only during the task execution. The context is not accessible during
 ``pre_execute`` or ``post_execute``. Calling this method outside execution context will raise an error.
 
+Accessing context variables in decorated tasks
+----------------------------------------------
+
+When running your callable, Airflow will pass a set of keyword arguments that can be used in your
+function. This set of kwargs correspond exactly to what you can use in your jinja templates.
+For this to work, you need to define ``**kwargs`` in your function header, or you can add directly the
+keyword arguments you would like to get - for example with the below code your callable will get
+the values of ``ti`` and ``next_ds`` context variables.
+
+With explicit arguments:
+
+.. code-block:: python
+
+   @task
+   def my_python_callable(ti, next_ds):
+       pass
+
+With kwargs:
+
+.. code-block:: python
+
+   @task
+   def my_python_callable(**kwargs):
+       ti = kwargs["ti"]
+       next_ds = kwargs["next_ds"]
+
+Also sometimes you might want to access the context somewhere deep the stack - and you do not want to pass
+the context variables from the task callable. You can do it via ``get_current_context``
+method of the Python operator.
+
+.. code-block:: python
+
+    from airflow.operators.python import get_current_context
+
+
+    def some_function_in_your_library():
+        context = get_current_context()
+        ti = context["ti"]
+
+Current context is accessible only during the task execution. The context is not accessible during
+``pre_execute`` or ``post_execute``. Calling this method outside execution context will raise an error.
+
 
 What's Next?
 ------------